Dual ingress ISP setup vs Juniper SRX

[tirexxerit]

Hi,

   there is a fundamental difference between Juniper SRX and Palo Alto Firewalls regarding how reverse route look up occurs

for a session. With Juniper SRX, if I have two ingress traffic via two different ISPs, I can put each into its own routing instance

and Juniper forwards the reverse traffic back via ingress ISP. It is very simple and elegant solution I think.

   However on PAN, reverse traffic e.g SYN+ACK requires another routing lookup which ruins this scenario as look up occurs

on another routing instance and response exits via what ever the default gateway is.

As far as I can see PAN has "enforce symmetric return" which is way limited per platform so you can't rely on this or some PBF

but it isn't comparible to what Juniper does. I wonder if there is anything on the roadmap or any other workaround which we can accomplish this on PAN. Hoping it isn't wishful thinking.

thanks.

[Remo]

Hi @tirexxerit

For roadmap information you need to talk to your SE and you also have to sign a NDA with PaloAlto Networks. But even then the chance is not really high that you will get the information you ask for. If you intend to buy 20 PA-7080 where you would need this juniper-behaviour/feature, then the situation probably looks a little bit different 😉 😛

But back to your problem: In what way is the enforce symwtric return future limited. From what you describe in your post, to me it seems like this feature would do exactly what you ask for. Or I don't fully understand your post.

On a paloalto firewall you could also configure the two interfaces in to different virtual routers which would eliminate the need for PBF (but requires some additional steps if the two virtual routers both connect to the same networks)

[pulukas]

Not sure I'm reading your topology correctly but I think you can do the same in PAN as you have in the SRX.

PAN also supports having the second ISP in a virtual router and if that is setup then the route lookup for the return path would stay on the same ISP as it does with your SRX configuration.

I think you might have the Policy Based routing method for dual ISP setup on the PAN instead of the separate VR?

[tirexxerit]

Hi,

   Here is a simplified topology. You have a server which may need to receive request

from two different IP blocks (each on two different ISPs) and PAN does DNAT towards

this server. I am saying I can do it on SRX because when I put ISP1 and ISP2 on two different VRs

SRX does the reverse look up on the VR that the first packet arrived hence flow already knows

how to return it back via ISP2 or 1

   but on PAN, reverse routing lookup is not done on the ingress VR. At least this is what I can

see on the flow trace. So on PAN if SYN packet entered via ISP2 and if PAN's default gateway is ISP1 then

serverA's SYN+ACK exits via ISP1 not ISP2. 

   As for symmetric return, correct me if I am wrong but this is based on keeping ARP per source address.

I tested this and when you receive a request from external IP, it puts an ARP entry in its PBF pool and it is

limited per platform. So if you exceed your ARP capacity you are done. (I believe so)

Via PBF, I can only instruct PAN to return serverA's packets via ISP2 instead of default gateway but this doesn't

give me the flexibility I have on SRX i.e to respond via each ISP simultaneously.

In this scenario, you are saying I should be able to achieve it?

thanks.

[Remo]

Hi @tirexxerit

Do you know how many IPs need to connecr over one specific ISP? You are right about the capacity per platform. On a PA-220 this number is 750. So it depends how many concurrent sessions you have.

Do you need the have the IPs on ISP1 available on ISP2 when ISP1 fails and do you need the server to see the real source IP? If this isn't a requirement you could assign dedicated VRs for each ISP and NAT the connection on the PA, so intenally you don't get routing problems for the reverse traffic - in case the capacity of return mac entries isn't enough on the platform you use/plan to use to support the amount of concurrent connections you need.

[tirexxerit]

As the numbers grow and even the current source addresses are more than our current platform, it isn't really scalable actually.

I am not sure about server side but NAT on the VR was one of the options but I think it is not a nice one. There may be 300-400K sessions which we may need to source NAT (which might grow as well)and when there is an issue about NAT or pools that will bring more operational complexity I think. 

From the discussions, I can see there is no built-in solution and the only automated workaround is NAT on the VRs

if we don't touch the server side. I will also reach out to PAN as well to see if they can implement such feature like SRX other than symmetric return 🙂

In the past, I had found SRX's behavior of reverse look-up design a bit odd but now I see in this particular scenario, it is near perfect.

thanks.

[pulukas]

Thanks for the diagram.  I think you are verifying you are using PBF.  You will need to remove PBF and configure the PAN the same way you do the SRX.

PAN also allows you to create virtual routers and put the second upstream in a separate VR rather than use the PBF method for dual ISP.

[tirexxerit]

PBF is for to solve a particular problem but I just made a test in my lab without PBF

which confirms my findings. Below is an ICMP request reply. ICMP request

creates the session and route lookup is done on the ingress VR3(ISP1)

and return packet's route lookup is done on VR1(default) and goes to a different gateway/ISP than

the packet initially arrived.

If putting both ISPs on their separate VRs worked, return packet should have done the route lookup on VR3

not on VR1 right?

 I am using PBF if I want to manipulate default routing decision which is required in some scenarios.

== 2018-09-03 14:44:43.858 +0200 ==
Packet received at fastpath stage, tag 59941, type ATOMIC
Packet info: len 102 port 18 interface 264 vsys 1
  wqe index 81832 packet 0x0xc001156400, HA: 0
Packet decoded dump:
L2:     00:0c:29:80:5c:29->00:0c:29:20:13:9f, type 0x0800
IP:     155.1.1.2->192.168.234.3, protocol 1
        version 4, ihl 5, tos 0x00, len 84,
        id 28245, frag_off 0x4000, ttl 63, checksum 42118(0x86a4)
ICMP:   type 8, code 0, checksum 59428, id 1619, seq 109
Flow fastpath, session 59941
IP checksum valid
2018-09-03 14:44:43.858 +0200  pan_flow_process_fastpath(src/pan_flow_proc.c:3548): SESSION-DSCP: set session DSCP: 0x00
Forwarding lookup, ingress interface 264  
L3 mode, virtual-router 3
Route lookup in virtual-router 3, IP 192.168.234.3
Route found, interface ethernet1/3.235, zone 1, nexthop 192.168.235.6
Resolve ARP for IP 192.168.235.6 on interface ethernet1/3.235
ARP entry found on interface 256




== 2018-09-03 14:44:43.868 +0200 ==
Packet received at fastpath stage, tag 59941, type ATOMIC
Packet info: len 102 port 18 interface 256 vsys 1
  wqe index 80605 packet 0x0xc006970700, HA: 0
Packet decoded dump:
L2:     00:0c:29:80:5c:29->00:0c:29:20:13:9f, type 0x0800
IP:     192.168.234.3->155.1.1.2, protocol 1
        version 4, ihl 5, tos 0x00, len 84,
        id 8324, frag_off 0x0000, ttl 63, checksum 30228(0x1476)
ICMP:   type 0, code 0, checksum 61476, id 1619, seq 109
Flow fastpath, session 59941
IP checksum valid
Forwarding lookup, ingress interface 256
L3 mode, virtual-router 1
Route lookup in virtual-router 1, IP 155.1.1.2
Route found, interface ethernet1/1, zone 2, nexthop 10.1.1.1
Packet forwarded to different zone 2 than zone 7 in session 59941
Packet dropped, forwarding does not match security rule

Packet dropped, pan_flow_ingress_parse error

[pulukas]

Just do a source nat to interface on the inbound traffic flow on vr3.  The the return traffic in vr1 will come back to vr3 match the session and return via the same ISP.

[tirexxerit]

thanks for the feedback Steve. We had discussed this but not sure there maybe some scalibility issues on 500K sessions

and need to check if servers have any dependency on source IP or not.

[usomal]

symmetric return ARP entry is for next-hop only. Why do you need to add ARP for all those ISP side IPs?