topic Re: Site to Site VPN Tunnel is up, but no traffic pass through in General Topics

Site to Site VPN Tunnel is up, but no traffic pass through

UXPSystems — Thu, 06 Apr 2017 18:17:55 GMT

Hi all. I am trying to setup a site to site VPN tunnel with one of our customer. I've got the dedicated layer 3 zone, tunnel interface, IKE Gateway, Virtual Router etc. configured per the Palo Alto admin guide. In the "IPSec Tunnels" section, it shows the VPN tunnel is up. However, I cannot access any of the server located at the customer's environment.

In the Traffic monitor tab, it shows the traffic is sending over to the customer's network, yet nothing is returning from them (Bytes Send = xxx; Bytes Received = 0; Packet Send = xxx, Packet Received = 0).

Am I missing something here?

Thank you.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

ansharma — Thu, 06 Apr 2017 18:29:34 GMT

Hi UXPSystems,

I'd say either the reply packet is getting dropped or we are simply not receiving any replies.

Take pcaps with filters:

1 - x.x.x.x - y.y.y.y

2 - y.y.y.y - x.x.x.x

The numbers '1' and '2' are the 2 rows you will create in the packet filter. The addresses x.x.x.x and y.y.y.y are the source and destination (and back) for the actual IPs you are pinging from and to. Configure packet capture for the drop, receive and transmit stage. Refer to this KB for more information on pcaps - https://live.paloaltonetworks.com/t5/Learning-Articles/How-to-Run-a-Packet-Capture/ta-p/62390

If we are simply not receiving packets, then the issue could be return route on the remote site. If we are receiving packets, then we'd have to check in the counters and flow basic (debug logs) to find out where it's going.

Additionally, select more colums in the traffic logs, like ingress and egress interfaces, etc.

Regards,

Anurag

Re: Site to Site VPN Tunnel is up, but no traffic pass through

TranceforLife — Thu, 06 Apr 2017 20:49:01 GMT

Worth checking the below article:

https://live.paloaltonetworks.com/t5/Management-Articles/How-to-Troubleshoot-IPSec-VPN-connectivity-issues/ta-p/59187

> show vpn flow name - check encap/decap bytes

Just some additional info!

Re: Site to Site VPN Tunnel is up, but no traffic pass through

UXPSystems — Fri, 07 Apr 2017 15:34:34 GMT

Hi. The PCAP Receive filter file shows bunch of TCP Retransmission to the target server. Wondering if the traffic doesn't know how to reach to the target ....

Re: Site to Site VPN Tunnel is up, but no traffic pass through

TranceforLife — Fri, 07 Apr 2017 15:44:20 GMT

Hi,

Packets received by the firewall from your side, right? Do you have a static route configured pointing to the tunnel interface to reach another end subnet?

FIB lookup.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

bradk14 — Fri, 07 Apr 2017 15:42:41 GMT

is it a PA on both ends of the tunnel? or what device are they using? dunno if it's of any help, but just throwing it out there because there is the requirement of the use of proxy ids in order to establish a tunnel with a policy-based vpn device to essentially identify the interesting traffic.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

TranceforLife — Fri, 07 Apr 2017 15:45:54 GMT

@bradk14 another good point. Phase 2 definitely would not be up if proxy ID mismatch.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

UXPSystems — Fri, 07 Apr 2017 16:46:04 GMT

I am not sure about the VPN product that the other end is using. As a test, if I configured the Proxy ID, the tunnel status goes into "down" state (red).

A static route existed for the remote network

If I do a tracert to the remote server, the tracert stops at our PA firewall.

The PA traffic monitor will show packets has send to the remote network, but no packet receives (eg: no return traffic).

I am wondering if the remtoe network didn't configure their route properly. Otherwise I don't see any issue on the VPN tunnel side, it is up, just no traffic coming back. Also the tracert result bothers me, as it shows stopped on our PA.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

Raido_Rattameister — Fri, 07 Apr 2017 18:22:42 GMT

If you don't configure ProxyID in Palo it falls back to default ProxyID and sends over 0.0.0.0/0

ProxyID is needed only if you connect to device that does policy based vpn instead of route based vpn.

Most likely other end does not have route back to you.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

TranceforLife — Fri, 07 Apr 2017 18:35:35 GMT

@Raido_Rattameister @UXPSystems as you said seems like other side having an issue to return traffic. You also can check encapsulation counters:

> show vpn flow name - check encap/decap bytes

Will give you a very good indication that the traffic is encapsulated and sent through the tunnel.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

UXPSystems — Mon, 10 Apr 2017 16:52:14 GMT

Hi all.

So I ran the "show vpn flow name" command, and I can see the "encap/decap bytes" field logs data there, and no authentication errors. The tunnel looks perfectly fine. Pretty much stuck on this one ...

Re: Site to Site VPN Tunnel is up, but no traffic pass through

TranceforLife — Mon, 10 Apr 2017 17:32:51 GMT

You don't run this tunnel between Azure on PAN-OS 7.1.3 or previous versions, do you?

Re: Site to Site VPN Tunnel is up, but no traffic pass through

UXPSystems — Mon, 10 Apr 2017 21:23:02 GMT

Hi. Nope, not running on Azure. It looks like the customer end is running on Juniper firewall if that helps.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

TranceforLife — Mon, 10 Apr 2017 21:29:45 GMT

Let's ask kindly @santonic

Re: Site to Site VPN Tunnel is up, but no traffic pass through

ansharma — Mon, 10 Apr 2017 22:54:10 GMT

Did you check the reply traffic route in the routing table of the peer (Juniper)?

Take pcaps on the peer and see if it's receiving packets from PA and/or sending anything back.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

UXPSystems — Tue, 11 Apr 2017 13:17:50 GMT

Hi all. I don't have access to customer's network, thus no pcacp available from their end. It is a pain to get a hold of their staff to investigate this issue. They keep responding they've done the necessary configuration at their end, and from their prespective, this VPN configuration has been completed at their end. Yet no network traffic coming from their end.

I've gone over the troubleshooting step as outline here,

https://live.paloaltonetworks.com/t5/Management-Articles/How-to-Troubleshoot-IPSec-VPN-connectivity-issues/ta-p/59187

I can see the "Encap Bytes" value keep increasing, while the "Decap Bytes" stays constant. Per the explaination, that indicate network traffic is going out, but nothing returns from the target.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

TranceforLife — Tue, 11 Apr 2017 13:59:00 GMT

I think in this situation you and another end should go for a conference call and get all troubleshooting steps alive rather than do a finger-pointing. Without the other end, it is going to be hard to get this resolved as you can see.

Re: Site to Site VPN Tunnel is up, but no traffic pass through

santonic — Wed, 12 Apr 2017 06:23:53 GMT

From what you described here i would say that the issue is definitelly on the other side. If you are sending packets into tunnel, they are getting them. Now they must find out what happens with them. And i feel your pain. I've also have many occasions when i was debugging VPN on the other end as well even tho i had contract only with my customer 🙂

One (very unlikely) issue could also be SPI missmatch; exchange info about SPIs with admin on the other end and check if they match.

I guess on your side you have only 1 pair?

Re: Site to Site VPN Tunnel is up, but no traffic pass through

UXPSystems — Mon, 17 Apr 2017 13:23:40 GMT

Hi all. After couple of email exchange with the customer, and conference calls. It ends up their network administartor configured the VPN tunnel, yet he didn't configure the routes for this VPN connection. That explains why the tunnel is up, but no traffic goes through. Now they've assigned another network administrator to look into, yet he is on vacation till mid May.... Nothing can be done till mid May. Thank you for all the feedback, much appreciated!