PA-450 Internet Traffic stops after 15 minutes

[Keatsie]

Replacing 2 Cisco M400 firebox with PA-450, mirrored the NAT and security polices from the M400 onto the PA-450. When we switch over the firewalls we get internet traffic both ways as expected but after 15 minutes all in and outbound internet traffic stops. Switch back the Cisco routers and traffic flows normally again, swap back the the PA-450 firewalls and 15 minutes later all internet traffic stops again.

We have been through with the ISP can checked the incoming routers config and the ISP is not blocking or stopping the internet traffic. We have simplified and switched of HA and running a single inbound service, same thing happens - doesn't seem to matter what we change NAT, Policy etc traffic stops roluting after 15 minutes?

Has anyone any suggests where to start looking outside of the obvious we have tried?

[JayGolf]

Hi @Keatsie ,

When traffic stops working, are you seeing anything in the monitor tab? or do you not see anything hitting the Palo? 

Could you clear the ARP cache at upstream devices? I've had issues in migrations when switching boxes and the upstream device was sending it to the MAC registered to the old device that was switched out. 

[Metgatz]

Hello @Keatsie 

Hello, the M400 Fireboxes are Watchguard Firewalls, not Cisco.

Well now my question is the following, when you talk about the HA switchover, do you mean the HA of the PA-450, of the ones you already migrated or this problem you have in your migration window, when you just remove the M400 and now you put the Palo Alto ?

This 15 minutes delay that you mention, may be due to the adjacent equipment entries, the ARP entries, that take time to update.

I mean the ARP table of an ISP router, for example, knows that for example you have IP 200.200.200.200.10 MAC 00:1B:44:11:3A:B7 which is the Outside Interface example of your Watchguard M400 Firewall. Now when you go and put in the Palo Alto, maybe the computers are not refreshing the ARP table in the most expeditious way.
What happens in these cases, it is recommended to clear or clear the ARP table or the ARP entry that references the IP against the Firebox MACs so that it quickly adds the PA-450s.

Also if there are switches in between, it is ideal to clear the MAC tables entry that references the Watchguard MACs to quickly add the new PA-450 MACs.

This in general nowadays is very transparent, however it often depends on each vendor and/or manufacturer.

Regards

[Keatsie]

Thanks for the response.

I can still see traffic in the monitor, the NAT and security plocies are still incrementing. I have restarted the pre and post firewall switches and I get the same response.I spoke with the ISP and have the configuration for the fibre terminating router and there is nothing on the ISP side that service monitoring.

Next steps, pull all the switch config files and check for any message, ICMP or MAC routing that is expected and missed

[Keatsie]

Thanks for the response, and you are correct the M400 are watchguard - my generalisation was incorrect.

I removed the HA from the config completely to simplify the network layout and routing, just to get a stable in and outbound traffic. I went back to basics, allow everything in and out - checking for consitent IP traffic, traffic still stops routing at the same time interval.

Having check the ISP terminating routers, there logs do not show any port blocking and they are not monitoring in away that would block traffic. I reset the pre and post firewall switches and cleared the ARP caches and still the same response. So I am now pulling all the switch configs to look for any monitoring etc.

[OtakarKlier]

Hello,

Enable logging on the PAN and check the logs. See if there is anything is in the block-IP log. My guess is that there are no blocks and this is an ISP issue (but ONLY a guess). 

Regards,