Perimeter FW in A/P HA directly connected to Palo Alto vwire in A/A HA

[MasterBubba]

the basic topology is:

Internet FW are non-palo alto in HA A/P directly connected (no switch in between) to a pair of Palo Alto in A/A HA -in Vwire mode (no Layer 3 on the Palo Alto but in transparent-zone)

then Palo Altos in A/A HA are connected to the Core switch where the rest of the environment connects to as well.

The internet FW will send a GARP in the event of a failover event. the VIP and associated mac address will be swapped between the standby and active units.

Issue:

#1 - When the perimeter FW in A/P HA failsover due to the internet interface issue (not the interface directly connected to the Palo Alto), we are noticing internet outages access from the users connected on the Core switch even though we validated the internet is up and working from the Internet FW, the failover process took place with issue.

#2 - We are also noticing the Internet FW session table reaching its maximum capability of 65535.

Questions:

#1 - #is it possible that the Palo Alto have not been made aware that the Perimeter Internet FW to which it is directly connected has undergone a failover? Does it even care or should it care

#2 - #How does the Core switch downstream the Palo Alto know to send its traffic to the Palo Alto (PA2) which is directly connected to the previously standby FW unit but now is the active FW (FW2 )due to the failover.

#3 - #Is it possible that the Palo Alto (again in vwire mode) will continue sending traffic to the now standby FW (FW1) instead of the active FW (FW2)?

#4 - In this setup, what role do the ARP, GARP, Mac address entries and updates continue to play an important role to ensure proper and optimal path selection all the way from the end-user devices connected to the core switch to the Internet FW via the Palo Alto?

FW1 <---> FW2

Active  --   Standby

   |                   |

   |                   |

PA1 <---> PA2

Active  --   Active

   |                   |

   |                   |

-------------------

One Core L3 Switch

-------------------

I am new to Palo alto but after doing a show mac all and show arp all the Palo Alto had zero entries listed in the output so it does not look like they keep track of arp or mac addresses either from the core side or upstream FW pair in Active/Standby HA mode.

Thanks for the help.

[OtakarKlier]

Hello,

Do you plan on using just he Palo Alto firewalls and removing the older HA pair? Do you need to run Active/Active as this can cause issues, also you have active/passive in front of them.

Regards,

[MasterBubba]

Thx for the reply . . .  As of now, there is no plan to retire the Internet FW in HA A/P.  The  PA HA A/A pair downstream to the HA A/P Internet FW was implemented a while back and based on what I am told the issue outlined started after the implementation of the PA HA A/A, but I rather prefer to understand what could be the root cause of the issue than assume anything.  I do agree with you that the setup does not make things easy to get to the bottom of the issue.  Since, yesterday's post, I can confirm that the PA does not have a policy which would filter out arp or garp traffic coming from the upstream internet FW during a failover event.

[MasterBubba]

If you know of a similar setup experiencing a similar issue please do let me know or anything documentation wise to read up on, I would welcome it as well.  So far, I have not been able to find this golden nugget of information 🙂

[OtakarKlier]

Honestly I would start by simplifying what you have, however I understand that may not be an option. Make the Palo Alto firewalls Active Passive to match the others and see what happens.

[CCIE11129]

I'm trying to come up to speed on vwire for a similar setup. The difference is that I have dynamic routing to the internet FW to/from the internal network.

Is your connectivity issue temporary? I was thinking that dynamic routing would give better failover, but I believe I just read that in a PA active/active setup active sessions that were originally passed through PA1 and now rerouted to PA2 due to FW1 to FW2 failover would be sent from PA2 to PA1 via the HA3 link. This would then send to the standby FW1 and dropped. I would assume new connections would be fine which is why I asked if your issue was temporary after a FW failover.

[CCIE11129]

Just read another forum that says for vwire in active/active that traffic that comes into port A of a PA that isn't the session owner that the traffic is sent to the owner via HA3, processed, and then sent back via HA3 to the receiving PA to send out the port B vwire interface. If this is the case, then I think this may work with dynamic routing between the internet FW and the inside network.

[MasterBubba]

Hi CCIE11129,

Would you happen to have the forum link post where you read up on the traffic being sent between the PA via its HA link due to being in active/active?  This is what we were thinking might be happening today when talking about the issue with my colleagues.  Regarding your question about being temporary, it is hard to answer because the fix has been to do a quick failover back to the original active FW1 once we realized that the session table was getting filled up with what appears to be embryonic sessions.  We also had to wait for the session time out to kick in for the session table to clear itself up.  We are suspecting that the session table was getting filled by traffic being sent to the FW1 (then in standby) for some reason instead of FW2 (then active) but we don't know if it is the Palo Alto doing this or if the downstream L3 core switch sending the traffic to PA1 instead of PA2.  The later would point to the mac address mapping being off from the L3 Core switch standpoint (this is why I am wondering the FW1/2 are doing the proper GARP when failing over).  the vWire config is comprised of only x2 ports so from the PA point of view if traffic is received on one port it should technically have only x1 port to use to send the traffic out, but as you mentioned there is the HA link to consider in an active/active setup.

[MasterBubba]

I hear you and agree with you 100%, unfortunately it is going to be a tough sell right now and to sell it well i need to prove it is the best solution to our pickle 🙂

[CCIE11129 · Accepted solution]

Here's the link https://live.paloaltonetworks.com/t5/general-topics/about-active-active-on-vwire/td-p/46863

If your FW is sending GARP and it's not getting through the PA, it concerns me that dynamic routing updates might not make it through the PA during a failover of the upstream FW. Seems like both should work based on the explanation at the link. Not sure if sessions are actually built for broadcast and multicast traffic though. I don't think they are for multicast if you don't enable multicast processing. Not sure about broadcast though.

[MasterBubba · Accepted solution]

Thanks for the update.  Since, I don't manage the PA appliance I have requested the admin to open up a case with PA TAC to get a better understanding of how it is configured and the expected behavior during a failover of the upstream HA A/P FW.  As far as I know, the routing IGP update are working and it looks like the issue seems to be a L2 port mapping between the Core L3 SW and the HA A/P FW via the PA A/A pair.  Once I know more, I will post my findings for the benefit of the community.
Thx to all for pointing me in the right direction.