AWS VM Series Gateway Load Balancers not working

[craig.beamish]

Hi All

Has anyone else had a play with the GWLB on AWS?
I know it must be PAN-OS 10.0.2 or higher to work,
I have tested with multiple instances, 
As a bump in the wire it works fine. until you apply NAT, then it doesn't work at all for any traffic that is NAT'd. 
I have an open TAC for this, they are replicating the fault to work it out but surely this was all tested before it went public.

I also found overlay routing breaks traffic flow. its not documented anywhere that I could find but what I found was it processes the GENEVE traffic in the virtual router where without it, is just an in-return non routed flow. 

If you've tinkered with it and actually got inbound/outbound NAT and/or overlay routing to function, please let me know what you did. 

sadly the documentation just doesnt provide any decent clarity for this feature.

Also extremely disappointed they havent integrated this into version 9.1.
I am hopeful they will add it with 9.1.7 in a functional state as I am not planning to move my clients to 10.0 until the list of known issues is about 1/4 its current size.

[jmeurer]

It had not been committed to a version yet.  You should reach out to your SE to track the progress internally.

[EsakkimuthuGanesan]

I am also facing the same issue. PA somehow it sees non-SYN packet and it drops it . If i disable TCP reject non-SYN temporarily then the application works. Not sure why PA is dropping the packet or why it sees  non-SYN packet

the Inbound traffic pattern is 

Internet -> ALB -> ( GWLB EndPoint Service -> GWLB -> PA FW -> GWLB -> GWLB EndPoint Service ) -> TGW -> Webserver

😞

[craig.beamish]

I thought i saw on 10.0.5 or 10.0.6 release notes that GWLB was working with overlay routing, is it still not fully functional?

[craig.beamish]

So now overlay routing is out and "functional" i've done more testing,

Its only feasible for single direction flow:

in > out 

or out > in
what ever interface has the VPC endpoints attached MUST be the interface for return traffic (i.e. VPCe Ethernet1/1.1, 'trust' network behind ethernet1/1)

so reading the docs:
traffic comes in network client > GWLB > VPC endpoint > GENEVE int ethernet1/1.1 > egress eth1/2 > nat/IGW > server
return: server > in nat/igw > eth1/2 > ethernet1/1 > client
the docs indicate that return traffic does not egress the geneve sub interface but rather the normal physical interface.

You can't have traffic come in eth1/1.1 and egress eth1/1 it just doesnt seem to work oddly.
  

[DLeskov]

Has anyone seen traffic coming on the ethernet1/1 interface instead on the subinterface. I'm running version 10.0.8 and it seems it happens randomly. Sometimes traffic appears coming from the private zone attached to the subinterfaces (which is mapped to the vpc endpoint) and some times the traffic appears on the Internal sec zone mapped to the ehternet1/1 interface.

[ibhattic]

Pings from my internal host to 8.8.8.8 or the external interface of the firewall are not working. Even curl -v https://www.google.ca does not work. I don't know if I need to make a change in the routing table or somewhere else I can't figure to make it work. I am playing with 10.1.4 for my setup.

[LukaszW]

Hi All,

I had something similar. This may help you all. 

Have a look on RFC for GENEVE protocol https://datatracker.ietf.org/doc/rfc8926/

Review your logs in AWS as you may be blocking GENEVE 6081 port. Its similar to IPsec how it works so you have to allow 6081 both ways. Destination 6081 will always be the same, what I mean return traffic will be to also 6081(not to random port)

Hope that helps.

Good luck and best regards,

Luke