Dual ISP, PBF and DMZ

[santonic]

Hello.

I have a specific question about certain situation. There is a customer with 2 ISPs, let's call them ISP1 and ISP2. Customer has a single PA device to which both ISPs are connected. Each ISP provides a block of public IP addresses which are routed to the PA device. With PBF we'll make a rule, which sets ISP1 as primary ISP and ISP2 as secondary. So default route will be set to ISP1 when ISP1 is up and to ISP2 when ISP1 is down. So far it's all pretty standard situation.

Now let's say teh customer has some server in its DMZ zone. Server has a local IP address in DMZ zone so NAT is needed. Customer wants the server to be available on BOTH ISPs all the time, Therefore we make 2 NAT rules; one for each ISP.

Let's say ISP1 is active at certain moment so default route is pointing towards ISP1. But someone tries accessing server over ISP2. In that case packet comes from ISP2, gets translated to private IP address, enters DMZ zone, server replies, packet reaches PA device again and now the PA device has to put packet on proper interface. In this case; will PA put packet on ISP1 interface where default route is pointing, or will it know that packet originated from ISP2 and will correctly put it on ISP2 interface?

Ok, I know this sitation can be avoided with proper use of DNS records or BGP, but it's a temporary solution so I'm really interested what PA will do in exactly such situation.

[mikand]

Good question... I hope someone from PA could answer if PBF will overrule any internal "express forwarding" (or what we might call it) or if the "express forwarding" will overrule the PBF for returning traffic.

I think something like this might work:

For your DMZ use some private ip range (just an example): 10.0.0.0/24 (in real life you would probably choose a smaller range than /24 aswell :smileysilly:).

Setup a D-NAT for incoming traffic from ISP1:

srczone: ISP1

dstip: 1.1.1.0/24 (or whatever ip/range it might be)

dnat: 10.0.0.0/24 (the DMZ ip/range)

Setup a D-NAT for incoming traffic from ISP2:

srczone: ISP2

dstip: 2.2.2.0/24 (or whatever ip/range it might be)

dnat: 10.0.0.0/24 (the DMZ ip/range)

if the above wont work (lets assume since your PBF for ISP1 vs IPS2 will overrule which path the return traffic will take) you can add:

Setup a PBF for outgoing traffic for ISP1 from DMZ:

srczone: DMZ

srcip: 1.1.1.0/24 (now I assume that NAT will be done before PBF)

action: Forward (1.1.1.254, or whatever ip nexthop for ISP1 have)

Setup a PBF for outgoing traffic for ISP2 from DMZ:

srczone: DMZ

srcip: 2.2.2.0/24 (now I assume that NAT will be done before PBF)

action: Forward (2.2.2.254, or whatever ip nexthop for ISP1 have)

[santonic]

Yep, in first part you describe NAT for incoming traffic just the way i was planning to do it.

But I don't think 2nd part for outgoing traffic will work. Packets originating from DMZ will have source IP of DMZ and destination IP some random public IP outside of company's network. So you can't distinguish packets by ISP there.

Unless!!! (Your example mentioning source NAT gave me an idea.)

When I do destination NAT translation for incoming traffic (from public to private IP) for each ISP i can do source NAT at the same time. That way source address of packet will connect packet with correct interface. So server will reply to that translated address and hopefully PA will handle the traffic correctlly then. Any thougths if this would work?

[mikand]

The 2nd part depends on which of PBF and NAT is done first internally.

If NAT is done before PBF then it will work - otherwise it will fail.

Using a SNAT/DNAT combo might work but it will add complexity to your solution :smileysilly: (and you will lose statistics in the server logs regarding which client did what on your servers and so on).

Another method could be if you could avoid NAT and set your servers in DMZ to be multihomed.

Lets say ISP1 gives you 1.1.1.0/24 and ISP2 gives you 2.2.2.0/24 and from this you use the first /29 for DMZ.

This way server1 at DMZ would have ip's:

1.1.1.1/29, defgw 1.1.1.6
2.2.2.1/29, defgw 2.2.2.6

This way we wont need to worry about if NAT or PBF is being done first since all thats left is PBF to take care of in your PAN.

Not only that you wont need to do DNAT for incoming traffic but the returning traffic will be much easier to setup for the PBF:

Setup a PBF for outgoing traffic for ISP1 from DMZ:

srczone: DMZ
srcip: 1.1.1.0/29
action: Forward (1.1.1.254, or whatever ip nexthop for ISP1 have)

Setup a PBF for outgoing traffic for ISP2 from DMZ:

srczone: DMZ
srcip: 2.2.2.0/29
action: Forward (2.2.2.254, or whatever ip nexthop for ISP2 have)

I think you could then also remove the PBF you already have and setup which ISP to use in your vrouter configuration instead (set ISP1 with lower metric than ISP2 if ISP1 is the prefered one for clients to use - PBF will override whatever vrouter says).

[rmonvon]

Hi...In our next release (summer 2012) we will have a new feature called 'Symmetric Return' or commonly referred to as Return-to-Sender.  This feature will ensure traffic coming from ISP1 will be responded via ISP1 and traffic coming from ISP2 will be responded via ISP2.  Thanks.

[santonic]

Thank you for your answer. Any workarounds for this which are available now?

[santonic]

@mikand

Yep, having multihomed server with public address in DMZ would solve this. But at this point we are not allowed to be changing any IP addresses at customer.

[Support_CC]

Assuming the server in DMZ has an IP address 10.0.0.11/24. Add and alias/additional IP on the server: 10.0.0.12.

Then do:

DNAT: Public1->10.0.0.11

           Public2->10.0.0.12

PBF:  10.0.0.11 to !LAN -> ISP1

          10.0.0.12 to !LAN -> ISP2

! - negate

This should work fine, complexity is on the Palo side.

best regards

rafal weglarz

[santonic]

Yep, that would work. However customer is ok with having his service up only on primary link when primary ISP is up so no complications needed atm

[mikand]

Or combine "my" idea with the one which www.cc.com.pl presented?

Let the server use whatever private ip it currently uses but add public ip's from each ISP as subinterfaces on the servers?

On the other hand I dont know how many subinterfaces the PA device will support without going for VLAN tagging.

[Support_CC]

@mikand I think that the server will not handle outgoing traffic properly. Having two default gateways configured doesn't mean that both are used. I don't think that any system responds to traffic using the same network interface as the one on which the traffic arrived (ignoring routing table).  So you end up with spoofing like behaviour from the FW's perspective. You would need PBR on server as well.

[mikand]

Perhaps adding some iptables magic would do it but I think its to overwork the problem in this case if you go that far 😃