Tunnel Monitor - PAN-OS SDWAN

[Clint_Phelps]

I've had an issue recently where randomly I've had members of my VPN mesh start to have the tunnel monitors going up and down constantly which causes BGP to never be able to establish with the peer since the static routes to the loopbacks are pulled from the route table.  This has happened randomly to 2-3 sites back to either of the hubs.  When the SD-WAN module creates all of the ipsec tunnels it addresses them from a pool you configure (10.254.0.0/16) for example.  Then it sets up the tunnel address on the other side as the monitored IP.  Since tunnel monitoring isn't subject to the normal flow of the data plate (NAT, etc) and is sourced from the local tunnel interface to the IP of the remote interface, as long as phase 1 and 2 are established there shouldn't be anything that can cause the tunnel monitor to fail as reachability would always be there.

I'm struggling to find a remedy to this.  We haven't upgraded firmware or the sd-wan module since well before this started occurring. Most of the branch firewalls are on 10.2.7h3.  One hub is on 10.2.6 and the other 10.2.4-h4. Reboots of the firewalls do not resolve the issue.  Has anyone else encountered this before?

[OtakarKlier]

Hello,

I've seen issues with routing protocols not being able to establish neighbors when a circuit is having issues. Our solution at the time was to down the interface of the circuit associated with the circuit until the provider stabilized it. Not sure if there is a flap detection threshold for a situation such as yours. Would be a useful tool to have.

Regards,

[Clint_Phelps]

Thanks.  I'm not seeing any issues on the circuits these tunnels are terminating at but I will look a little deeper and see if there is some debugging I can turn on that may show more info that I'm missing.  The firewall shows the tunnels up with active SAs on phase 1 and 2.  No other apparent issues other than the monitor itself going up and down constantly.  Like every few seconds.