Application incomplete Site to Site VPN

[inzamam.shahid]

Greetings,

I wish to run an issue that one my sites is experiencing with a site to site VPN. The issue that is experienced is that some applications mainly mail application will show up in the logs as incomplete. I will aim to give you the full picture of this so you can understand the setup and hopefully advise of a solution.

(Working Site)SITA A – Aruba controller that has a site to site VPN connection to a PALO. Between the PALO and the Aruba controller there is a MPLS network for this site. The applications are identified correctly and everything is working fine. This site uses a 10.10.x range for clients. The traffic is NATTED correctly and no asymmetric routing is detected.

(Not working site)SITE B – Aruba controller is physically connected to a PALO, which has a site to site VPN connection to another PALO through an MPLS network. The application here get identified as incomplete when they hit the first palo which is directly connected to the Aruba controller. This site uses a 10.11.x range for clients, the traffic is NATTED correctly and no asymmetric routing is detected. It just some applications get identified as incomplete.

I have dropped the MTU size between the tunnels and this has made no difference.

I have compared the traffic range of 10.10 and 10.11 and both are the same, but 10.11 shows the traffic as being incomplete.

I have created an any any rule for a test user and still it does not work for them.

I have tried to use application over ride, but application does not work correctly.

Just want to know if you guys have similar problems and hopefully can advise me going forwards. I can draw up a quick network diagram if required.

Many Thanks,

[reaper]

on the topic of your MTU adjustment, did you also set the TCP MSS adjust ? : TCP MSS adjustment for IPSec traffic

this may assist in forscing a lower MTU on the remote end

if you then stil lsee the issue, try setting up a packetcapture and verify if the remote end is not resetting the MSS (i had a case once where another device in the middle was also adjusting the mss to 1500 while we only had 1400 available)

the behavior you describe sounds similar: the remote end was ubaware of the adjusted MTU and kept sending packets with an mtu of 1500 and dont-fragment set, causing them to be discarded

[inzamam.shahid]

Thanks for the reply.

I have dropped the MTU sizess at both ends of the tunnel to no ovail. I have tried 1350, 1300 & 1200. The same values were set at both ends of the tunnel each time.

I looked into the TCP MSS feature and it looks like it needs to be applied to the physical interface and not the tunnel. Just to clarify are you suggesting that we set an MTU size like 1300 on the tunnels and then set TCP MSS to 1500 to the external interface with the GP config.

[OtakarKlier]

Hello,

Almost everytime I have seen the application show us as incomplete, its because the tcp 3 way handshake did not complete. So it turned out to either be rounting, dns resolution, or a rule blocking traffic. 

I'm sure you have already done this, however try tracing from source to destination and vice versa. Check the logs, traffic, threat, etc., for anything that is being blocked. Since you are NAT'ing between the sites, I would also make sure the appropriate NAT rules are being hit.

Hope this helps.

Regards,

[inzamam.shahid]

Hello, 

I have done some PCAPS on this and can see that the three-way handshake does complete. After the three way handshake the connection is being reset from the destination, the three way handshake happens again completes and the dest then resets the connection again. 

I am trying to get a capture from one of the working sites so I can compare how the packets are being handled, but do you know why trying to send traffic down a VPN tunnel why would the dest reset the connection?

[OtakarKlier]

Just curious, why nating over the VPN. Not saying its wrong, but I have done this in the past and caused issues with voice traffic ( turned out the PBX we were using and the soft-phones didnt like to be natted). Not saying its the cause of your issues, just trying to add some insight to my experiences.

[inzamam.shahid]

I think I resolved this now finally. There is no natting happening over the VPN. Natting is happening after the traffic has gone down the VPN and then going out the external interface of the FW. 

To resolve this I had to do numerous packet captures and have dropped the VPN at either side to 1200 MTU and set service to any on the FW that is directly connected to the Aruba controller.