Static route path monitor shows UP with invalid next hop

[StephenBuck]

I'm running PAN-OS 10.1 on a VM-100. I have DHCP on an interface and use a script to update an address object with the default gateway from the DHCP interface. I have a static route with next hop set to this address object and path monitoring enabled. I've run into a situation where if the DHCP lease expires (something upstream fails with the provider), then the address object gets set to 0.0.0.0. The path monitor still shows up when using the next hop of 0.0.0.0.

I can reproduce this easily by creating a static route to 9.9.9.9 with next hop of 254.254.254.254 and it shows down. With a next hop of 0.0.0.0 it shows with a status of UP.

Is this expected behavior?

[BPry]

@StephenBuck,

I'd open a TAC case and report the behavior, as this is more than likely an issue with how the firewall is interrupting the 0.0.0.0 address in the monitoring process. 

[aleksandar.astardzhiev · Accepted solution]

Hi @StephenBuck ,

Interesting case...But it seems that @BPry is in right direction.

It seems that Linux/Unix based systems are interpeting the 0.0.0.0 as localhost (127.0.0.1), while Windows return failure:

You should be able to test it by yourself - run ping from your firewall to 0.0.0.0 and you will see replies from localhost.

So I don't believe it is problem of path-monitor - as it is just simple ping packets to the destination you configure.

You probably should consider modifing you script to identify if the return ip is 0.0.0.0 to take different action, instead of updating the path-monitor with that value.

[StephenBuck]

Thanks @aleksandar.astardzhiev I'm going to be checking for null or 0.0.0.0 and replace it with 254.254.254.254 which seems to work. In the meantime, I've opened a TAC case and waiting to see what they say. Would definitely require a code change in PAN-OS to prevent replies to 0.0.0, or prevent 0.0.0.0 as a next-hop, and in the latter case, my script would fail to commit a 0.0.0.0 and so would need to be modified anyway.

[StephenBuck]

TAC advised that this is expected behavior since the underlying Linux pings to 0.0.0.0 are succeeding. I've verified with my script that setting the next hop to 254.254.254.254 will cause the path monitor to fail when there is no DHCP address on the interface.

[johnwalshaw]

For the DHCP interface you should uncheck the option to auto create default route. This is because the static route monitoring is for a static route, not for an auto generated route.  Therefore, once you uncheck the option to auto create default route for DHCP interface, you need to create a static route to the ISP gateway IP.  Only issue is that if the ISP ever change the gateway IP, your static route would become invalid.

[StephenBuck]

I forgot to mention that I did turn off the option to auto insert the default gateway from DHCP. My script simply queries the DHCP interface to get the gateway handed out by the DHCP server and I set the next hop to this address. That's what started all this since the script would put in a 0.0.0.0 for the next hop when there is no address on the interface (DHCP lease expired and there was no connectivity with the ISP).

[johnwalshaw]

I'm re-reading the thread and love that you have a script to harvest the gateway IP from the DHCP interface. The issue of your lease expiring is interesting as that should renew at the lease half-life.  That is the problem I would perhaps think about.  The process to trigger (schedule) the lease renew might be worth looking at. But is this an ISP issue at the core also?  Regarding what the target is that you ping, there might be an opportunity there.  Here is my config as feel it is representative of the internet.

[johnwalshaw]

Also, forgot to mention that I have each ISP on a separate virtual router, therefore there is no backdoor zero route possibility via another ISP link. The result is that path monitoring to specified internet destinations is working without fault.

[StephenBuck]

One of my ISPs has a lease time of only one hour. The Palo renews just fine as long as there isn't any issue between it and the DHCP server. If there's a short outage that lasts longer than an hour, that's when I noticed a problem before changing my script. When the lease expired and the Palo was unable to renew a new IP, the interface would show 0.0.0.0 for the next hop gateway.

[StephenBuck]

I do the same thing and have my ISPs in their own virtual router and use BGP to advertise the default route to the internal VR since you can't peer between VRs using OSPF.

[johnwalshaw]

How do you have path monitoring configured? Maybe post a screenshot similar to what I posted?