Dual ISPs with 2 PAN Firewalls

[dan_dalessandro]

I am in need to set up a branch office with Dual ISPs and PAN firewalls.  Everything I've seen & read detail the process with a single PAN and PBF's.   Has anyone accomplished the same with 2 PANs??  I am trying to get the set up in my head & having issues obviously.    Regards, Dan

[sraghunandan]

Hello,

Have you had a chance to look at this doc:-

https://live.paloaltonetworks.com/docs/DOC-1608

[sdurga]

Hi Dan,

You can try Active Active HA. Here is a doc that explains this behavior. So in the doc u can see the two upstream routers to each pan, you can assume them as your ISP's and can relate your situation to this doc.

https://live.paloaltonetworks.com/docs/DOC-1756

Thanks,

Sandeep T

[mikand]

What about using each PA box as a single device and use Panorama to configure them equaly?

Perhaps more complex solution (since the router between your internal net and PA must decide which way to use) but will bring you twice the performance compared to using active/passive or active/active.

[dan_dalessandro]

Mikand (hopefully I have your name right), when you say using each device as a single device & using Panorama to configure them equally, how would the setup look ?  How would failover work?

[mikand]

It depends on which equipment each ISP has at your end (CPE) but in short I meant:

1) Configure each PA as a single device (no HA) - the only thing that will differ is basically just the ip addresses on the interfaces.

2) Use Panorama to make the security policies etc equal on both PA's (otherwise you could of course manually configure each PA but each time you reconfigure the security policies you then must do the work twice).

3) In the routers before (and after) your array of PA boxes you will use ECMP to make the routers loadshare the traffic through your PA boxes.

Dont forget to use "ip load-sharing per-destination" (which is default but still) which will make sure that a srcip+dstip combo will always use the same physical path (until that fails).

ECMP means that the router have two (or more) nexthop for a particular route with equal cost/metric.

For example:

ip route 0.0.0.0/0 nexthop <PA1> metric 10

ip route 0.0.0.0/0 nexthop <PA2> metric 10

The above will bring you twice the performance in total (but not for a specific client since a specific client will only use one PA at a time) compared to active/passive or active/active (because in active/active the traffic is still owned by just one of the boxes which will be the limit of total performance).

[sdurga]

mikand so in this case we are doing load sharing on the routers rather than on the Paloalto and sending the shared traffic to Paloalto's right ? so I believe it will essentially boil down to same Active-Active cofig where both the devices are sharing the half load or am I missing some thing here ?.Also curios about how the failover condition will work in this case because the upstream/downstream routers do not know the failure of the PAN devices.

[mikand]

Yes, the routers will do the loadsharing (decide which PA will get the traffic/session).

A more fancy solution is to use a true loadbalancer such as F5 or so but since most routers supports ECMP and ECMP is enough for this case I see no reason to invest in two (well four for redundency) F5's for this matter.

Compared to active-active config each PA will not know that there is another PA in the network. Another difference compared to active-active is that with active-active, even if both PA are active - only one "owns" the session which gives that the total throughput (or number of concurrent connections) is the same as with active/passive.

On the other hand, this limit (active/active wont give you 2x performance - only 1x) is good when one box fails because you have the same throughput with only one still functional box.

However... PA-5060 is rated for 10Gbit throughput (threat prevention) which means if you need lets say 40Gbit throughput PA currently doesnt have any option for that (as a single box). This is where an ECMP setup can be handy:

1) Setup ECMP in the routers before and after the PA array.

2) Connect 4x PA-5060 individually configured (no HA).

3) Done! 40Gbit in total throughput (but still max 10Gbit per session - same as when using ether/portchannel/lacp).

There are other "loadsharing" methods one can apply in such situations (depending on how your network is setup) - one is to statically "loadshare" based on VLANs instead.

Like VLAN 1-999 goes through PA-5060_1, VLAN 1000-1999 through PA-5060_2, 2000-2999 through PA_5060_3 and 3000-4096 through PA-5060_4 (or which VLAN id is the highest that PA supports).

The drawback here will instead be if you have for example your fileserver and mailservers at VLAN 101 and 102 which might be like 90% of your total utilization this traffic will slam into PA_5060_1 while the other three boxes will basically just idle (compared to an ECMP setup where all four would work).

The good thing on the other hand is that you wont get a decrease in bandwidth if one box fails (well given the PA-5060_x is actually two boxes running in active/passive).

Regarding failover condition ECMP will react if the interface goes down and then push that traffic over one of the other paths (in this case it depends on how you have setup your PA regarding tcp-reject-non-syn if the moved sessions will fail or continue to function) - otherwise you can use IP-SLA (or similar) to withdraw the route if, for example, a ping doesnt reach to the other side of the PA for a particular path.

For example a setup like:

Outer-Router:

int0/1: VLAN101 10.0.101.2/30

int0/2: VLAN102 10.0.102.2/30

ip route x.x.x.x/x next 10.0.101.1 track 1

ip route x.x.x.x/x next 10.0.102.1 track 2

PA-boxes

int1: VLAN10x 10.0.10x.1/30

int2: VLAN20x 10.0.20x.2/30

Inner-Router:

int0/1: VLAN201 10.0.201.1/30

int0/2: VLAN202 10.0.202.1/30

ip route 0.0.0.0/0 next 10.0.201.2 track 1

ip route 0.0.0.0/0 next 10.0.202.2 track 2

The IP-SLA config could be something like (example for Inner-Router):

ip sla 1

icmp-echo 10.0.101.2 source-interface int0/1

timeout 1000

threshold 2

frequency 3

ip sla schedule 1 life forever start-time now

track 1 ip sla 1 reachability

ip sla 2

icmp-echo 10.0.102.2 source-interface int0/2

timeout 1000

threshold 2

frequency 3

ip sla schedule 2 life forever start-time now

track 2 ip sla 2 reachability

This way Inner-Router will use each physical interface to ping the equal interface at Outer-Router and by that be able to detect if the PA in this particular path is functional or not (and if not then remove the route connected to this particular ip-sla through track).

For more information on IP-SLA: http://www.firewall.cx/cisco-technical-knowledgebase/cisco-routers/813-cisco-router-ipsla-basic.html

What I havent verified yet is if track will enable ECMP or not (if both track 1 and 2 are successful, will both routes have the same metric and therefor have ECMP enabled?).

[sdurga]

What I havent verified yet is if track will enable ECMP or not (if both track 1 and 2 are successful, will both routes have the same metric and therefor have ECMP enabled?).

Yes if both the tracked objects are up then both the routes will be up and load balancing will happen.

Regarding the performance

Operating two PAN's in standalone might give you 2xperformance but that might not be true in all the cases.

I think the real problem comes in when a device fails .

When one of the PAN in a cluster of two fails and we are able to route all the traffic of this PAN to the other working PAN via IP-SLA tracking and making the changes to the existing routes on the routers, now the one working PAN should be able to handle 2xperformance which is not possible.

So if you need 40Gig  throughput then connecting 4x5060 will give you that performance but if one of them fails then there is no room for the other 3x5060 to take any of that extra load no matter whether we are doing any kinds of load sharing (assuming all the boxes are running at the max capacity).

Also interesting case would where if we are doing destination NAT on pan for internal server. In such case HA will be the only solution.

[dan_dalessandro]

thanks for your time & input.  much appreciated.  I will read over the notes above but I wanted to be clear here:  There are no Cisco routers involved in the architecture.  It would simply be ISP1->PAN1->Core Switch.  Then ISP2->PAN2->Core switch.  Yes, the core switch is a single point of failure, but apparently the customer is OK with that.

[mikand]

ECMP is available from most manufacturers nowadays.

The core switches in this case... are they perhaps L3-switches? 🙂

[dan_dalessandro]

yes it's a 4506

[mikand]

If you have updated IOS and fairly new supervisor it should support ECMP according to:

Release Notes for Catayst 4500 Series Switch, Cisco IOS XE 3.1.x SG - Cisco Systems

"

8-Way CEF Load Balancing Yes Yes Yes

"

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps4324/product_bulletin_c25-553133.html

[dan_dalessandro]

Might I be able to get away with an active/passive employing IP SLA on the core 4506 ?  Relatively confident we will not need or achieve 40Gb throughput.. Bear in mind, this is all new to me.

Re: Dual ISPs with 2 PAN Firewalls