topic Re: IPsec Site-to-Site VPN trouble (decap bytes 0) in General Topics

IPsec Site-to-Site VPN trouble (decap bytes 0)

AMS-IX — Sat, 08 Aug 2015 06:06:20 GMT

Hi all.

I am trying to set up an IPsec s2s tunnel with non-Palo Alto peers. So far I have tried 3 different peers (Strongswan 5.3.2, Cisco router, Cisco SOHO router) and every time I have problems seeing incoming decrypted traffic to the PA.

"Local site" being the PA one, here's the info I have so far:

- IPsec tunnel is up

- "show session all filter protocol 50" shows one active tunnel session (for the ipsec tunnel)

- "show vpn flow tunnel-id name <tunnel name>" shows encap packets, but no decap packets

- Proxy-ID is set to the local NAT address range after translation has been done, and to the native address range for the remote site (no NAT is being done there)

- Remote site end hosts receive packets from the local site, via the tunnel (e.g. echo requests from the post-NAT IPs. NAT range is specific to the PA tunnel interface)

- Local site end hosts never receive replies (e.g. echo replies)

- I have tried putting the (internal) tunnel interface in both the "internal" zone, as well as the "vpn" zone, no luck

- I am using a loopback as the external interface, set in the vpn zone

- Policies from vpn to internal zone and vice versa allow all traffic

UPDATE (some additional notes):

- IPsec tunnel is terminated in a logical loopback interface on the PA, which is configured in the VPN zone.

- Although all policies have logging enabled at session end, I never see logs of tunneled packets incoming from the other peer

- I have thought of configuring the IPsec tunnel to terminate on a logical interface in case it's the loopback interface causing the problem, but all external physical interfaces are set on the untrust zone and I would like to keep

VPN and untrust zones/policies separate.

- The PA is directly connected via a VLAN to our two edge routers. The edge routers have an ingress L3 ACL permitting esp/ahp packets towards the PA loopback address. Removing the L3 ACL entirely also did not help.

Any advice/insight would be greatly appreciated!

Message was edited by: Aris Lamprianidis

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

pulukas — Sat, 08 Aug 2015 10:23:45 GMT

Based on this information I believe the issue is on the other device. Since you vpn shows decap of zero, this means no packets are coming out of the tunnel from the remote side.

If the PA were dropping or blocking by policy or inspection, you would see decap increment but nothing on the local host packet capture.

What do the IPSEC stats for the tunnel show on the remote node?

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

pankaku — Sat, 08 Aug 2015 10:42:19 GMT

Could you paste the output of show session id <id number> for the traffic going from PA to Remote site.

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

AMS-IX — Sat, 08 Aug 2015 12:33:15 GMT

Hi Steven. That was my assumption on the first take. Then however I configured the two other peers (Cisco SOHO and Strongswan) and I see the same behavior..

Adding to my reply, the packet counter for the SA from the Strongswan side *is* increasing (see line before last). This part of the output of "ipsec statusall":

Security Associations (1 up, 0 connecting):
      PA[1]: ESTABLISHED 20 minutes ago, 200.200.159.51[200.200.159.51]...100.100.139.9[100.100.139.9]
      PA[1]: IKEv1 SPIs: 46112c603ecc9b01_i 5c5827074640bb75_r*, pre-shared key reauthentication in 7 hours
      PA[1]: IKE proposal: AES_CBC_256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/MODP_1536
      PA{2}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: c9cb52d8_i d369e77c_o
      PA{2}:  AES_CBC_256/HMAC_SHA2_512_256, 68712 bytes_i (818 pkts, 220s ago), 70644 bytes_o (841 pkts, 1s ago), rekeying in 20 minutes
      PA{2}:    172.26.16.0/22 === 10.20.30.0/25

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

AMS-IX — Sat, 08 Aug 2015 12:51:28 GMT

Hi Pankaj, here you go (output redacted/sanitized):

"100.100.139.9" is the PA

"200.200.159.51" is the StrongSwan peer

10.20.30.0/25 is the post-NAT PA network

172.26.16.0/22 is the StrongSwan network

aris@PA(active)> show session all filter protocol 50


--------------------------------------------------------------------------------
ID          Application    State   Type Flag  Src[Sport]/Zone/Proto (translated IP[Port])
Vsys                                          Dst[Dport]/Zone (translated IP[Port])
--------------------------------------------------------------------------------
183171       ipsec-esp      ACTIVE  TUNN       100.100.139.9[51659]/vpn/50  (100.100.139.9[59260])
vsys1                                          200.200.159.51[21208]/vpn  (200.200.159.51[54121])

And this is the output of "ip xfrm policy" on the StrongSwan side:

# ip xfrm policy

src 10.20.30.0/25 dst 172.26.16.0/22

dir fwd priority 2887

tmpl src 100.100.139.9 dst 200.200.159.51

proto esp reqid 1 mode tunnel

src 10.20.30.0/25 dst 172.26.16.0/22

dir in priority 2887

tmpl src 100.100.139.9 dst 200.200.159.51

proto esp reqid 1 mode tunnel

src 172.26.16.0/22 dst 10.20.30.0/25

dir out priority 2887

tmpl src 200.200.159.51 dst 100.100.139.9

proto esp reqid 1 mode tunnel

src 0.0.0.0/0 dst 0.0.0.0/0

socket in priority 0

src 0.0.0.0/0 dst 0.0.0.0/0

socket out priority 0

src 0.0.0.0/0 dst 0.0.0.0/0

socket in priority 0

src 0.0.0.0/0 dst 0.0.0.0/0

socket out priority 0

src ::/0 dst ::/0

socket in priority 0

src ::/0 dst ::/0

socket out priority 0

src ::/0 dst ::/0

socket in priority 0

src ::/0 dst ::/0

socket out priority 0

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

AMS-IX — Sat, 08 Aug 2015 15:44:51 GMT

Update: I've also tried setting up a s2s with a Fortigate device (not my configuration), and I still see the same issue.

Also note that the StrongSwan - Fortigate IPsec tunnel works OK.

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

pankaku — Sun, 09 Aug 2015 08:52:41 GMT

Hello AMS-IX,

Initiate a ping from the host sitting behind the PA firewall to the host sitting behind the other side. There after check the session for that ping

show session all filter application ping source <> destination <>

show session id <>

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

AMS-IX — Mon, 10 Aug 2015 06:50:29 GMT

Hi Pankaj,

Here's the output:

aris@panix2(active)> show session all filter application ping source a.b.c.11

--------------------------------------------------------------------------------

ID Application State Type Flag Src[Sport]/Zone/Proto (translated IP[Port])

Vsys Dst[Dport]/Zone (translated IP[Port])

--------------------------------------------------------------------------------

333341 ping ACTIVE FLOW NS a.b.c.11[59216]/int-servers/1 (10.20.30.11[59216])

vsys1 172.26.16.6[328]/vpn (172.26.16.6[328])

aris@panix2(active)> show session id 333341

Session 333341

c2s flow:

source: a.b.c.11 [int-servers]

dst: 172.26.16.6

proto: 1

sport: 59216 dport: 39

state: INIT type: FLOW

src user: unknown

dst user: unknown

s2c flow:

source: 172.26.16.6 [vpn]

dst: 10.20.30.11

proto: 1

sport: 39 dport: 59216

state: INIT type: FLOW

src user: unknown

dst user: unknown

start time : Mon Aug 10 08:41:47 2015

timeout : 6 sec

total byte count(c2s) : 102

total byte count(s2c) : 0

layer7 packet count(c2s) : 1

layer7 packet count(s2c) : 0

vsys : vsys1

application : ping

rule : IS-V-001

session to be logged at end : True

session in session ager : False

session updated by HA peer : False

address/port translation : source

nat-rule : Internal Servers(vsys1)

layer7 processing : enabled

URL filtering enabled : False

session via syn-cookies : False

session terminated on host : False

session traverses tunnel : True

captive portal session : False

ingress interface : ae2.105

egress interface : tunnel.2

session QoS rule : N/A (class 4)

tracker stage firewall : Aged out

end-reason : aged-out

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

pankaku — Mon, 10 Aug 2015 14:25:16 GMT

Here traffic is not coming back to the PA firewall.

You can try doing PCAP on the PA firewall and then check where firewall is receiving the packets. Monitor>Packet capture> set two filters for a.b.c.11 to 172.26.16.6 and 172.26.16.6 to a.b.c.11

then add all four pacp.

If you get drop pacp then it means firewall is dropping the packet if not then might be firewall is not receiving any packet.

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

AMS-IX — Mon, 10 Aug 2015 15:23:28 GMT

Thanks for your reply, Pankaj.

I already in fact did that: I can see the ICMP echo request in the cluster, but no ICMP echo replies are ever seen.. I have already opened a support request with our integrator. I was hoping this whole thing to be a misconfiguration of some sort, but chances of that are getting slimmer..

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

ArnljotSeem — Wed, 11 Oct 2017 12:13:41 GMT

Did you ever find a solution to this problem? I have the same issue with a StrongSwan, although a simpler setup without NAT. I'm using an external IP as my peer IP, so no loopback.

Ping works both ways, although not for every packet size. E.g. From the firewall (inside interface) size=1080 works but not 1085, size=282 works but not 283 !?

Traffic is flowing from local site, but no reply is ever received. I did a packet capture on the tunnel interface, and see the three-way handshake, but when our host does a http GET, I see no reply.

Very odd. We're running PANOS 7.0.14 and we have numerous other VPN's up and running, but only this one with Strongswan.

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

AMS-IX — Mon, 13 Nov 2017 19:33:41 GMT

Hello @ArnljotSeem,

Unfortunately this thread is now 2 years old, so I cannot recall what the root cause was. Based on your problem description so far though, I'm inclined to say that the issue behind it is not the same one.

You're saying that other VPN s2s solutions work, and this is specific to the PA and StrongSwan. I'd check release notes for any later StrongSwan versions, and/or try a different FOSS IPSec solutions, if at all possible. This message sounds hopelessly generic, but I'd at least wanted to let you know I didn't have the answer you were seeking for anyway 😕