AWS Multi-VPN Tunnel with Palo Alto NGFW - Flow Issue

[patrick.lim]

My PA NGFW managed to setup VPN tunnels with AWS VGW. AWS given 2 sets of VGW where each of the VGW comes with 2 links that will connect to NGFW 2 ISP link respectively with different set of public IP Address.. Below are the setup flow:

NGFW ISP1 -> AWS Tunnel1 (vgw1)
NGFW ISP1 -> AWS Tunnel2 (vgw1)
NGFW ISP2 -> AWS Tunnel3 (vgw2)
NGFW ISP3 -> AWS Tunnel4 (vgw2)

Tunnel1 will always be the first active tunnel. When the ISP1 down, all the VPN traffic will failover to tunnel3 and both sites able to communicate/ping each others. However, when the ISP1 and tunnel1 recovered, the hosts behind the NGFW were switch to tunnel1 to access to AWS remote VPC white the hosts from AWS did not swing back to tunnel1 which they're still sending traffic through tunnel3. Is there a way that we can address this at PA NGFW side? or it has to be done at AWS?

By the way, VPN setup are based on static routing.

[rmfalconer]

Any reason you aren't using BGP for this? There are ways to influence BGP so you can prefer specific tunnels in a specific order.

[patrick.lim]

the parameters given by AWS implementer based on static routing, if the BGP works in my mentioned environment. I don't mind to get the team to change go BGP. I have limited knowledge in BGP, Appreciate if you could share me more about this or example config steps both PA and AWS. Tq

[rmfalconer]

I implemented this over a year ago at a former job so I don't remember exactly where the documented steps are. I believe that within the AWS portal, after configuring BGP on the gateway, the exact BGP config will be generated for various models of devices. You just have to select Palo Alto and the specific CLI will be generated, including the necessary IP addresses to use. AWS assigns addresses from the APIPA range for both ends of the tunnel.

As I recall, there really isn't much modification for BGP on the AWS side so everything needs to be done on the PA. On the outbound updates to AWS, I did AS prepend to get the tunnel preference we wanted. For prefixes learned inbound from AWS, I changed the local preference to prefer tunnels in a specific order.

However, If you aren't comfortable with BGP, it might not be the best solution. It works great until it doesn't and you have to troubleshoot at 2am. 

Has the asymmetric traffic flow been a problem?

[patrick.lim]

Asymmtric flow was causing issue now due to the PA firewall drop the packet because of the different tunnel returned. It even happenned on ping traffic causes timed out. It was due to PBF & Static Routing metric. By now, BGP will be my last option to solve the issue. I did gone through the document from AWS - https://docs.aws.amazon.com/vpc/latest/adminguide/palo-alto.html and it seems to be straight forward configuration. By the way, since it is on BGP, do we need to involve anyting with ISP?

[rmfalconer]

The ISP won't need to be involved. The BGP sessions will be contained within the VPN tunnels.

[patrick.lim]

The PA Firewall NATed behind the F5 LB. Meaning PA Firewall do not hold public IP addresses at the interface. Will that works?

[rmfalconer · Accepted solution]

Yes, that's fine. BGP will run on the tunnel interfaces on the PA that connect to AWS. 

[licenselu]

Hello,

You can also use OSPF inside IPSec tunnel. Never touch AWS but since OSPF is almost available on any paltform, it should be supported...

Regards,

HA

[rmfalconer]

OSPF is not supported in AWS. Options are BGP or static.