Trouble routing VXLAN traffic as it enters the outside interface

[SteveBallantyne]

Hello community,

I am attempting to create a VXLAN over IPSec solution between my PA-3250 and a remote Fortinet FortiGate 61E. I have managed to get things configured correctly on the FortiGate (I think) as I am seeing the traffic entering on the Palo side.

I am using Tunnel Inspection on the Palo side and it appears to be set up correctly. In the Monitor > Tunnel Inspection on the Palo, the traffic can be seen there. Even though when I click the magnifier to take a deeper look on the flow the "Tunnel Inspection" checkbox is NOT checked, which seems strange to me.

I created a security rule for phone traffic and I am seeing that traffic pop into my standard Palo Monitor > Traffic > Logs. But the applications are all coming up "Incomplete". I am also not able to ping back and forth.

So it appears everything works - with the exception of the routing. And I am not sure how to fix that?

Has anyone here done multiple VLAN's over a VNI using VXLAN over IPSec? If so - how did you get your traffic to *route* correctly?

[reaper]

you should focus on establishing vxlan connectivity before looking into tunnel inspection

your sessions coming up as 'incomplete' means the vxlan 'tunnel' (VTEP) is not being set up properly between your local and remote switch, which makes tunnel inspection impossible

if you're not able to ping: make sure routing is set up properly into the ipsec tunnel interface for the remote subnet

then make sure security rules are set from your local zone to the tunnel zone, and vice versa, as needed (to the firewall vxlan/VTEP are UDP packets flowing back and forth over the ipsec tunnel)

once basic connectivity is verified, make sure both ends are able to establish the vxlan/VTEP connectivity

once that is established look into setting up tunnel inspection

hope this helps

[SteveBallantyne]

That is helpful, thank you. I am beginning to think that this is an issue on the Fortigate side.

Although the strange thing is that it looks like the packets are emerging from the Fortigate side "in tact" and the Palo is just dropping them. Here is an example ...

I have a test phone that I put a static IP onto, 10.50.80.10. It's on VLAN 580. Then I have a phone server on the Palo Alto side, 10.10.70.10. I ran a packet trace on the Palo, filtering for the phone IP. It seems that the packets are assembled with the correct VLAN and the correct VXLAN VNI of 1001. But they are dropped, and as noted the Tunnel Inspection is not being performed.

I have a case open with support - but not getting any feedback after the first 24 hours. 🙂

[SteveBallantyne]

I just got off of a Zoom session with PA support. They did some further digging into the session flows via the CLI and came to the conclusion that my packets are coming into the Palo, but the return path is broken on the FortiGate side of things.

Since the session is never fully establishing (no ACK), the tunnel inspection is not kicking into gear. That is, the *intent* to inspect is there, but the conversation is one-way.

I have already engaged FortiNet support - now it's a waiting game. I will share some information and configuration examples if we can ever sort this out, as I couldn't find a lot of useful documentation on their side. And I know there are many customers out there with Palo's in the main office, and "baby firewalls" at their remote sites.