How to Configure a Palo Alto Networks Firewall with Dual ISPs and Automatic VPN Failover

by panagent on ‎01-24-2013 10:38 AM (96,876 Views)

Overview

This document explains how to configure a Palo Alto Networks firewall that has a dual ISP connection in combination with VPN tunnels.

Configuration Goals:

  • A single device with two internet connections (High Availability)
  • Static site-to-site VPN
  • Automatic failover for Internet connectivity and VPN

Setup

This setup is frequently used to provide connectivity between a branch office and a headquarters.  ISP1 is used as the primary ISP on Ethernet1/3.  ISP2 is the backup ISP on Ethernet1/4.

Configuration

The configuration is identical on both firewalls, so only one firewall configuration is discussed. In this example, there are two virtual routers (VR).

Dual Diagram.png

Interface Configuration

Configure two interfaces:

  • Eth 1/3: 10.185.140.138/24 (connection to ISP1) in the untrust zone
  • Eth 1/4: 10.80.40.38/24  (connection to ISP2) in the untrust zone

Virtual Routers

There are two virtual routers:

  • VR1: Primary (ISP1) (Ethernet1/3)
  • VR2: Secondary (ISP2) (Ethernet1/4)

Each VR has an ISP Interface attached, but all other interfaces will stay connected to VR Secondary, as well as all future interfaces. The purpose is to let all interfaces be known by connected routes and routes on the VR as their routing method when the Main ISP goes down.

  • Primary VR has Ethernet1/3 interface attached

Primary Router GEn.PNG

  • The Primary VR routes include the default route and return routes for all private addresses back to the Secondary VR, where the actual interfaces are as connected routes. When the traffic is forced out the interface through the PBF, the traffic will know how to get back to the Secondary VR where the interfaces live.

Primary Routes.PNG

  • Secondary VR has the Ethernet1/4 attached with all the other interfaces, as shown below:

Secondary Router GEN.PNG

  • Secondary VR routes for all connected interface will show up on the routing table as connected routes, and the route for the tunnel will be taken care of by Policy-Based Forwarded (PBF).
  • To force the traffic out the Primary ISP interface, use the PBF Sourcing from the Trusted Zone:

PBF Source.PNG

  • The firewall tells the PBF not to forward traffic destined to a private network, since it cannot route private addresses on the Internet (as there might be private network addresses that need to be forwarded out). Click Negate.

PBF Negate DEST.PNG

  • As shown in the example below, set up the forwarding out of the Primary Interface, with monitoring to disable the rule, if the destination being monitored is not available. Revert the traffic to use the routing table of the Secondary VR where all connected routes exist.

Forward Tab complete.PNG

  • Configure a Source NAT policy for both ISPs. Make sure to define the destination interface on the "Original Packet" tab for both Source NAT rules.

Screen Shot 2015-02-05 at 4.52.14 PM.png

The reason for the multiple VRs is because both tunnels are up and running at the same time. If connectivity is to ISP1, it will failover to ISP2 as soon as possible. If the backup VPN over ISP2 is already negotiated, that will speed up the failover process.

Phase 1 Configuration

Gateways.PNG

For each VPN tunnel, configure an IKE gateway.

Phase 2 Configuration

Tunnels.PNG

For each VPN tunnel, configure an IPSec tunnel. On the IPSec tunnel, enable monitoring with action failover if configuring the tunnels to connect to anther Palo Alto Networks firewall. Otherwise, set up the PBF with monitoring and a route for the secondary tunnel.

Tunnel Monitoring (Palo Alto Networks firewall connection to another Palo Alto Networks firewall)

  • Primary tunnel with monitoring.

Primary Tunnel Monitor .PNG

  • Secondary tunnel with monitoring.

Secondary Tunnel Monitor.PNG

  • In Action, configure the Monitor Profile to Fail Over.

Tunnel Monitor.PNG

  • With this method, using tunnel monitoring there are two routes in the routing table, the first with metric of 10 for the Primary VPN traffic, and the second with the metric of 20 for the Secondary VPN. Since the tunnels terminate on the Secondary VR, the routes will be placed on that VR.

Routes for VPNs.PNG

Policy-Based Forwarding (Palo Alto Networks firewall connection to a different firewall vendor)

  • This method can be used when the connection is between two firewalls.
  • State from what Source Zone.

NONPA PBF.PNG

  • Indicate when the traffic is destined to the network on the other side of the tunnel (in this case it is 192168.10.0/24).

Destination PBF.PNG

  • Forward the traffic down the tunnel.

Forward PBF.PNG

  • When the PBF is disabled, because the destination is not reachable, the other VPN will start using the routing table with a route that has the same destination but is using the other configured tunnel.

Note: In the above example, a probe is sent out to 192.168.10.2 to check if it's reachable. The probe must have a source IP address and will use the IP of the egress interface, which will be the IP address of the interface 'tunnel.' If an IP address is not configured on the tunnel interface, the PBF rule will never be enabled. In this scenario, an arbitrary IP needs to be configured, such as 172.16.0.1/30. A static route for destination 192.168.10.2 must be added with next-hop as the tunnel interface. Otherwise PBF will always fail because traffic initiated from the firewall will not hit the PBF rule. Make sure the remote device knows how to return the packet. When working with a Cisco ASA, make sure it knows how to return traffic to 172.16.0.1/30. Additionally, configure a Proxy ID for this network on the Palo Alto Networks device's IPSec tunnel configuration.

owner: rvanderveken

Comments
by skrall
on ‎10-08-2013 02:42 PM

When testing connectivity, do not use any IP address owned by the firewall as one of the endpoints. Addresses owned by the firewall can not POLICY BASED FORWARDED.

by mario11584
on ‎01-28-2014 03:53 PM

"For each VPN tunnel, configure 2 PBF entries"

Looks like only one per tunnel was configured. Does the writer mean 2 PBF entries on each side of the VPN?

by santonic
on ‎12-09-2014 04:38 AM

What happened to the previous version of document? Is that one still ok to use? I find solution with 3 VRs more logical.

And shouldn't one of the tunnel interfaces be in Primary VR in this example?  Because one of the tunnels is terminated on IP address of Primary VR

by glasater
on ‎12-09-2014 04:47 AM

Use of three VRs is easily scaled from two with the same concepts that are with two. The use 3 VRs initially is excessive and unnecessary for a basic dual ISP with VPN failover. The example here is more logical to configure for most designs.

by santonic
on ‎12-09-2014 05:35 AM

What about tunnel interfaces? Shouldn't each tunnel interface be in the VR where the VPN is terminated? So one of the tunnel interfaces should be in Primary VR?

by glasater
on ‎12-09-2014 05:50 AM

No, All interfaces would terminate on the secondary VR. This makes it so that there is minimal configuration of routes needed. the Gateways can be created on the Untrust interfaces respectively, but the Actual Tunnel terminate on the Secondary VR with all other LAN, Tunnel and other interfaces and the firewall will already know where the connected routes are on that VR for the Traffic coming out of the tunnel, so hence no need for unnecessarily routes configured. The only routes in the Primary VR being return routes for private address as needed for traffic pushed out that interface.

by santonic
on ‎12-09-2014 05:59 AM

Ok, thank you for info.

Gonna configure it tomorrow and see how it goes.

by glasater
on ‎12-09-2014 06:02 AM

Yes. No problem. This what PA engineer here in our office use for configuration and is recommended. in cases where you might get or add a third ISP you can expand on this configuration easily. But for the majority of configurations there are only 2 ISPs. and the main reason this is recommended is for each VR to have the respective 0.0.0.0/0 route for traffic generated from the box to use to make sure we keep traffic symmetric.

by santonic
on ‎12-09-2014 06:32 AM

I will have to expand it a bit as well. FW on the other side (not PA) also has 2 ISPs so I'll have to make 4 tunnels to cover all scenarios.

by glasater
on ‎12-09-2014 06:37 AM

Since you are going to a Non-PA firewall tunnel monitoring has some additional requirements mentioned in the the following Doc:

Selecting an IP Address to use for PBF or Tunnel Monitoring

Also, a more detailed explanation on DPD and Tunnel Monitoring Doc is below here:

Dead Peer Detection and Tunnel Monitoring

These should help as well. On a side note, when thinking about traffic passing through the PA or sourced from it. PBF policy is only for Traffic traversing the Palo Alto. Traffic sourced from the Palo Alto will always use the routing table. Another reason for Separating ISP interfaces to their own VR in general. Especially for things like VPN gateway route to peers.

by santonic
on ‎12-10-2014 03:36 AM

Thanx for additional info.

Yeah, i know about restrictions with PBF and traffic originating from the device itself.

by traymondchia
on ‎03-26-2015 02:48 PM

Interesting document.

the only more or less important missing piece I can think of here is related to security policies for the scenario.

by santonic
on ‎03-30-2015 12:16 AM

Security policy is the same whether you have VPN redundancy or not. Only traffic routing is different.

I would also like to thank again for this document. It was helpful with my scenario as well: PA central location with non-PA remote locations, dual ISPs on central and remote locations, dynamic IPs with one of the providers on remote locations.

However sometimes comes a situation where both devices have IPSEC up, routing is correct, both devices are using same SPI-s and traffic still isn't going through VPN tunnel. Weird stuff.

by mkhavari
on ‎04-10-2015 08:37 PM

Thank you for this document. I followed your instructions on setting up the VRs, however when I fail to the secondary ISP, I was unable to ping an external host on the internet. I removed the static called "Primary ISP" from the Primary VR and added a static route on the secondary VR point to the secondary ISP and I was able to failover and failback between the two ISPs. Would this configuration work while following this document to complete the configuration?

by Thibaut_Vilminot
on ‎05-27-2015 09:43 AM

Hi,

Thanks indeed for this document.

However, I am not sure to see the point of the 2 VR, as most of the configuration is done on the Secondary VR.

Wouldn't it also work with only 1 VR?

by pulukas
on ‎05-27-2015 03:03 PM

There are two VR because we need to have two default routes, one for each ISP.  A single router can only have the one active default route.  By separating each ISP into their own router we can keep both links active and the routing tables complete.

by Thibaut_Vilminot
on ‎05-28-2015 07:34 AM

Hi Steven,

Thanks for your quick answer. However, why put 2 default routes, if the route to ISP1 will be handled by the PBF? I mist miss something.

by mario11584
on ‎05-28-2015 08:53 AM

Traffic sourcing from the firewall won't use PBF rules. You'll want static routes for both ISPs for traffic that sources from the firewall.

by Thibaut_Vilminot
on ‎05-28-2015 08:58 AM

Indeed, forgot about that! Thanks for the explanations!

by LeandroBarbosa
on ‎06-06-2015 07:39 AM

Hi,

I have problem with vrouter routing.

the following scenario:

I have two PA-200 at headquarters and branch, the headquarters has two wans and branch as well.

in the headquarters:

1/1 - wan Embratel

1/2 - wan Oi

1/3 - LAN

vrouter-Embratel with default gw

(Route 192.168.2.0/24 tunnel.8 metric gw 10)

vrouter with Oi-default gw

(Route 192.168.2.0/24 tunnel.9 metric gw 10)

tunnel.8 with vrouter-Embratel monitor ip 169.254.1.1

tunnel.9 with vrouter OI-monitor ip 169.254.2.1

vrouter-lan

(Route 192.168.2.0/24 gw next-vr vrouter-Embratel metric 10)

(Route 192.168.2.0/24 gw next-vr vrouter Oi-metric 20)

PBR

source zone: LAN

destination: 192.168.2.0/24

action: foward

Egress Interface: tunnel.8

profile: failover

next hop: 169.254.1.1

Monitor IP: 169.254.1.2

disable this rule .....

source zone: LAN

destination: 192.168.2.0/24

action: foward

Egress Interface: tunnel.9

profile: failover

next hop: 169.254.2.1

Monitor IP: 169.254.2.2

disable this rule .....

Policy:

LAN - VPN full acess

VPN - LAN full acess

in the branch:

1/1 - wan Embratel

1/2 - wan Oi

vrouter-Embratel with default gw

(Route 192.168.1.0/24 tunnel.8 metric gw 10)

vourter with Oi-default gw

(Route 192.168.1.0/24 tunnel.9 metric gw 10)

tunnel.8 with vrouter-Embratel monitor ip 169.254.1.2

tunnel.9 with vrouter Oi-monitor ip 169.254.2.2

vrouter-lan

(Route 192.168.1.0/24 gw next-vr vrouter-Embratel metric 10)

(Route 192.168.1.0/24 gw next-vr vrouter Oi-metric 20)

PBR

source zone: LAN

destination: 192.168.1.0/24

action: foward

Egress Interface: tunnel.8

profile: failover

next hop: 169.254.1.2

monitor ip: 169.254.1.1

disable this rule.....

source zone: LAN

destination: 192.168.1.0/24

action: foward

Egress Interface: tunnel.9

profile: failover

next hop: 169.254.2.2

monitor ip: 169.254.2.1

disable this rule.....

Policy:

LAN - VPN full acess

VPN - LAN full acess

I'm closing with the vpn wan Embratel headquarters with Embratel branch wan affiliate, the I do the same with Oi wan.

When the affiliate Embratel link falls redundancy for the link Oi works, though the heardquarters can access all affiliate resources except for the ping ip affiliate to PA-200 LAN, check the routing headquarters that route vrourter-LAN remains active by tunnel.8, in the case could not because the vpn fell.

vrouter-lan (headquarters) display by "show routing route" in CLI

192.168.2.0/24 gw next-vr vrouter-Embratel 10 (A)

192.168.2.0/24 gw next-vr vrouter-Oi 20

Why does it happen this issue in the Firewall kernel routing?

other resources does not fall by the network 192.168.2.0/24 due to OI PBR which is active.

by Rene
‎11-13-2015 07:30 AM - edited ‎11-13-2015 07:35 AM

What are those Monitor IP addresses for the tunnel monitoring (Palo Alto to Palo Alto connection)?

I see IP addresses 172.31.1.2 and 172.31.2.2 configured as Destination IP.

 

Are these arbitraty IP addresses setup on the tunnel interfaces to be monitored?

As I think these cannot be an IP address on the remote subnet as they are both in a different subnet(?)

 

And what about the configuratoin on the other end when it only has 1 internet line?

Do you need to create 2 IPSec tunnels on 2 different tunnel interfaces and setup 2 routes for the return traffic toward the 2 different tunnel interfaces, one with metric 10 and the other metric 20? 

by mali77
2 weeks ago

I have a similar situation but in my case both ISP's are connected to the same VR.  Can I accomplish the same if both are connectec to the same VR?

by
2 weeks ago

It can be accomplished but will grant you less flexibility. If you simply want to differentiate between outgoing traffic (eg bandwidth consuming browsing and mission critical applications), this will work just fine

by mali77
2 weeks ago

Thank you for the reply, is there any documentation for that by any chance?

 

Basically I have

 

SG3

ISP1 (eth1/7)

--------------> WAN-VR2

ISP2 (eth1/8)

 

Then I have a whole bunch of other sub interfaces on the LAN side:

VSYS3

trust1 eth1/24.1

trust2 eth1/24.2

trust3 eth1/24.3

 

Do I start with creating PBF under VSYS3?

 

Do I create a

 

 

by
2 weeks ago

The article above is the recommended setup, if you choose to only use 1 VR your options get limited to policy based forwarding

by mali77
2 weeks ago

I understand that however I am unable to set it up like that because of the way existing setup is.  I have a need to try to do it with both ISP's connected to the same VR.  My understanding is Palo can't just simply do multiple default routes.  And I also have the LAN side connected to a different VR so even if I create a secondary VR for the second ISP.  I will then need to move the whole LAN under the secondary VR.  Which means downtime and just can't do that right now.

by IT-Admin
2 weeks ago

We're looking to follow these directions. However, there's some steps that needs clarification, such as the secondary VR static routes to internal LAN. On the primary VR, it shows that the internal LAN subnets get routed to next VR (secondary VR). On the secondary VR, should we also set static routes to our next hop (to our core LAN router)?

Also, the secondary VR default route for "0.0.0.0/0" goes to 10.185.40.100, shouldn't this be to the gateway of the 2nd ISP? Unsure where 10.185.40.100 comes from in the example.

If the PBF is disabled, how does the internal LAN know to route through the secondary VR?

by
2 weeks ago

@IT-Admin : on your second VR the internal network would be a connected network, so no static routes are needed unless your internal network has a router. so in your case, yes you need additional static routes on VR2 for your LAN networks, pointing to the LAN router next hop

 

correct, that appears to be a typo, this should be 10.80.40.x

 

the second VR is connected to the networks, so if the PBF fails/is disabled, vr2 will be default for all traffic

Register now
Ask Questions Get Answers Join the Live Community
Contributors