topic Fail-over VPN site-to-site in General Topics

Fail-over VPN site-to-site

BigPalo — Thu, 07 Feb 2019 11:40:10 GMT

Hi,

We have a PA with two VPNs configured. VPN-Main is the active one and if this vpn falls, the traffic must go through the other VPN-backup. The fact is that when the active VPN falls, the route that has the Palo Alto continues going through the previous VPN, it does not refresh the route and adds it through the new tunnel.

This configuration worked when this deployment was done but suddenly stop working.

We have configured a tunnel monitor with destination IP and profile fail-over.

In VPN Main (active):

In VPN (backup):

And this is the route table:

tunnel.29 is the main (active). The metric is 2.

tunnel.27 is the backup. Metric is 5.

Why PA is not deleting the "active" route if the monitor ip is down through the vpn down.

Re: Fail-over VPN site-to-site

OtakarKlier — Thu, 07 Feb 2019 17:03:22 GMT

Hello,

Its because they are static routes. For this type of config I would recommend one of the following.

For VPN tunnel A

Use Policy based forwarding with the monitoring and "Disable this rule if nexthop/monitor ip is unreachable" checked.

Then use a static route for tunnel B

The PAN will enforce PBF policies prior to the routing table so when the PBF disables its self the routing table takes affect.

The second option is to use PBF for both tunnels and tunnel B lower on the list. The firewall reads these from top down so it will always prefer tunnel A unless it goes down then switch to tunnel B.

Hope that makes sense.

Re: Fail-over VPN site-to-site

BigPalo — Fri, 08 Feb 2019 09:03:33 GMT

Yes, but its not necesary PBF. It should be work only with tunnel monitoring

https://docs.paloaltonetworks.com/pan-os/7-1/pan-os-admin/vpns/set-up-tunnel-monitoring

Whats the problem with static routes??? static route should dissapear when monitor ip is down.

Re: Fail-over VPN site-to-site

LukeBullimore — Fri, 08 Feb 2019 09:20:55 GMT

Hi @BigPalo

The static route will be removed from the FIB not the RIB - if you go to the VR and click "More runtime stats" -> "Forwarding Table" it shouldn't be there.

Re: Fail-over VPN site-to-site

BigPalo — Fri, 08 Feb 2019 11:25:40 GMT

but the new traffic should use backup tunel, since the static route dissapear for active path.

Re: Fail-over VPN site-to-site

OtakarKlier — Fri, 08 Feb 2019 15:27:02 GMT

Hello,

In the static route, was the Route monitoring configured? Sorry if I missed it somewhere in the thread?

https://docs.paloaltonetworks.com/pan-os/8-0/pan-os-new-features/networking-features/static-route-removal-based-on-path-monitoring.html

Regards,

Re: Fail-over VPN site-to-site

LukeBullimore — Fri, 08 Feb 2019 19:39:05 GMT

Hey @OtakarKlier

I may be completely wrong here, but the way that I thought it worked was that if you had tunnel monitoring configured with action of 'fail-over' the actual failover mechanism works by removing any static routes associated with that tunnel interface from the FIB - effectively making it so you can start using another IPSec tunnel via routes in your RIB.

https://docs.paloaltonetworks.com/pan-os/8-1/pan-os-admin/vpns/set-up-site-to-site-vpn/set-up-tunnel-monitoring/define-a-tunnel-monitoring-profile.html#

Reading the documentation - when failover option is selected, tunnel interface is disabled so all routes with that tunnel interface are indeed removed from the FIB. So yes @BigPalo this setup is valid and the traffic would start using the backup tunnel so long as your primary tunnel has tunnel monitoring with failover option set.

Cheers,

Luke.

Re: Fail-over VPN site-to-site

BigPalo — Mon, 11 Feb 2019 08:14:57 GMT

PanOS is 7.1.x. It this version there is no path monitoring in the static route.

Re: Fail-over VPN site-to-site

BigPalo — Mon, 11 Feb 2019 08:16:23 GMT

So the version is 7.1.x. We think that its all configured properly but we dont know why its not working

Re: Fail-over VPN site-to-site

OtakarKlier — Mon, 11 Feb 2019 21:49:27 GMT

A ticket with support may help find the rason why.

Re: Fail-over VPN site-to-site

aleksandar.astardzhiev — Tue, 12 Feb 2019 09:37:17 GMT

Hi @BigPalo,

We have similar setup (IPsec failover to secondary tunnel) for two different firewall deployments. Both are on 8.1.x, but the first one was configured back when the FW was still on 7.1.x

I would say that your configuration seems OK - indeed you don't really need policy based routing (I still don't understand why there are instructions for doing such...). As @LukeBullimore mentioned the tunnel monitor will "disable", or bring down the logical tunnel interface, after that is is pure networking - any static routes assosiated with interface in down state will be removed from the FIB - there are still in RIB since they are static, but they are not active so they are not used to forward traffic.

The biggest "gotcha" with tunnel monitor is the encryption domain (proxy-IDs). I learned this through the hard way - If your tunnel is configured with multiple proxy IDs, FW will try to send pings to each of them. But in order for the pings to be successful your source and destination needs to match the proxy ID (interesting traffic needs to match the phase2 SAs)

For example:

ProxyID: local 10.10.10.0/24 - remote 20.20.20.0/24
tunnel.1 IP - 10.10.10.254/32 - tunnel monitor destination 20.20.20.254

The config above will work since you have only one ProxyID which is matching the source and destination addresses of the ping packet.

But if you have:

ProxyID-1: local 10.10.10.0/24 - remote 20.20.20.0/24

ProxyID-2: local 10.10.10.0/24 - remote 30.30.30.0/24
tunnel.1 IP - 10.10.10.254/32 - tunnel monitor destination 20.20.20.254

Here tunnel monitor will report that this tunnel is down. The reason for that is the FW will try to send ping for both proxy-IDs. Pings will success for the first one, but will fail for second proxy ID, because the destination is not matching the remote encrpytion. And all static routes assosiated with this tunnel will be inactive (not installed in the FIB)

So I would suggest to double check if the firewall is still believing that main tunnel is down:

- Under the GUI check tunnel status (the far right icon under IPsec Tunnels)

- Under GUI check statistics for tunnel monitor - show vpn flow name <name if your tunnel >

- If you have multiple proxy-id configured run above command for each proxy-id

Re: Fail-over VPN site-to-site

BigPalo — Tue, 12 Feb 2019 11:21:49 GMT

We dont use proxyIds. Its blank.

Re: Fail-over VPN site-to-site

aleksandar.astardzhiev — Tue, 12 Feb 2019 12:08:13 GMT

What is the output of the commad:

> show vpn flow name <name of your tunnel>

Do you see statistics for the monitor? What are they showing?

Re: Fail-over VPN site-to-site

Abdul_Razaq — Tue, 12 Feb 2019 13:58:13 GMT

Hi @BigPalo,

This is what i feel, i feel you may face little trouble with this configuration

You have a default route through tunnel and have a tunnel monitoring with fail over in primary and wait-recover in secondary.

and if you dont have a static route to remote peers other that through tunnel,

consider following scenarios

1- primary tunnel-monitor is up - everything works you wont face any issues,

2 - primary tunnel monitor is down - it makes tunnel disable and removes the route, second tunnel will become up and will have a default route through secondary tunnel. as you may not have a static route other than this to remote peer, primary monitoring might be keep on trying to rekey through seconday tunnel which may not be successful. what if your secondary tunnel aldo down now, as you have default monitoring(wait-recover), this tunnel will try to recover and route will be still there.

So i feel you should add a static route to remote peers not through tunnel, and no need of tunnel monitor in secondary as if first one succesfully rekeys, it will add a lower metric route.

Re: Fail-over VPN site-to-site

aleksandar.astardzhiev — Tue, 12 Feb 2019 15:15:12 GMT

@Abdul_Razaq,

From the screenshots in the original post you can see his routes through the tunnels are not default routes.

Re: Fail-over VPN site-to-site

LukaPecher — Thu, 25 Jul 2019 07:35:11 GMT

Hi @aleksandar.astardzhiev

We have the same problem with failover VPN. We have 6 proxy IDs in the tunnel and the primary ipsec tunnel drops. You explained why it drops and now I am interested if you have maybe any solution for that problem?

Re: Fail-over VPN site-to-site

aleksandar.astardzhiev — Mon, 29 Jul 2019 08:10:47 GMT

Hi @LukaPecher,

My solution was to split the proxy-id into separate IPsec tunnel configuration. It may sound very confusing, so I will try to explain it as clear as possible:

- You will need four ipsec tunnel - two tunnel to primary remote peer and two tunnel to secondary remote peer

- First two tunnels must be assigned with same tunnel interface (let say tunnel.1) and same IKE gateway (remote peer)

- Second two tunnels also must be assigned to same tunnel interface - tunnel.2 and same remote peer

- First primary tunnel will be configure with tunnel monitor enabled and with only one proxy-id that match the source and dest of the monitor

- Second primary tunnel will be configured with tunnel monitor disabled and with as many proxy-id as you like/need, excluding the proxy-id configured in the first primary

Identical you need to configure the second two tunnels to the secondary remote peer.

So under Network > IPsec Tunnels you should have something like that:

That way the tunnel monitor will send pings only via the correct proxy-id.

If the pings fail (for what ever reason), tunnel monitor will bring the logical tunnel interface. And because we are using the same tunnel interface for the second tunnel (to the primary peer without monitor), all the routes to tunnel.1 will be remove from FIB and traffic wil be redirected to secondary tunnel.

Re: Fail-over VPN site-to-site

treese — Thu, 18 Jun 2020 16:12:25 GMT

How would pbf work with VPN. I thought VPN traffic you could use pbf?

Thank you

Re: Fail-over VPN site-to-site

OtakarKlier — Thu, 18 Jun 2020 16:27:37 GMT

Hello,

Yes PBF can work. You would set the static route to the secondary tunnel and the PBF would point at the primary VPN with a tunnel monitor and disable of the policy.

The PBF takes effect prior to the Virtual Router, so if the PBF is disabled because of primary VPN down and tunnel monitor, the Virtual router then uses the static route that points at the secondary VPN tunnel.

Hope this helps.

Re: Fail-over VPN site-to-site

treese — Thu, 18 Jun 2020 16:38:41 GMT

Ok in my situation I have 1 VPN tunnel as backup and the primary fiber circuit lives on our core switchs. The VPN tunnel points to the VIP on our cores until I manually change routes on the Palo to move to vpn when fiber circuit failure occur. The other side isn't owned by us and don't support BFD. I have a flow direct set on cores to remove the primary route (when unavailable) and send the traffic to the FW to engage the VPN but the FW routes are static and trying to decide to use PBF if possible to switch the traffic. The VIPs on cores won't ever go down and looked at tunnel monitoring and path monitoring too but not sure how well it would work with only one VPN tunnel and the primary route to the core VIP. All thoughts are welcome. Thank you