IPsec Site-to-Site VPN trouble (decap bytes 0)

Reply
Highlighted
L1 Bithead

IPsec Site-to-Site VPN trouble (decap bytes 0)

Hi all.

I am trying to set up an IPsec s2s tunnel with non-Palo Alto peers. So far I have tried 3 different peers (Strongswan 5.3.2, Cisco router, Cisco SOHO router) and every time I have problems seeing incoming decrypted traffic to the PA.

"Local site" being the PA one, here's the info I have so far:

- IPsec tunnel is up

- "show session all filter protocol 50" shows one active tunnel session (for the ipsec tunnel)

- "show vpn flow tunnel-id name <tunnel name>" shows encap packets, but no decap packets

- Proxy-ID is set to the local NAT address range after translation has been done, and to the native address range for the remote site (no NAT is being done there)

- Remote site end hosts receive packets from the local site, via the tunnel (e.g. echo requests from the post-NAT IPs. NAT range is specific to the PA tunnel interface)

- Local site end hosts never receive replies (e.g. echo replies)

- I have tried putting the (internal) tunnel interface in both the "internal" zone, as well as the "vpn" zone, no luck

- I am using a loopback as the external interface, set in the vpn zone

- Policies from vpn to internal zone and vice versa allow all traffic

UPDATE (some additional notes):

- IPsec tunnel is terminated in a logical loopback interface on the PA, which is configured in the VPN zone.

- Although all policies have logging enabled at session end, I never see logs of tunneled packets incoming from the other peer

- I have thought of configuring the IPsec tunnel to terminate on a logical interface in case it's the loopback interface causing the problem, but all external physical interfaces are set on the untrust zone and I would like to keep

VPN and untrust zones/policies separate.

- The PA is directly connected via a VLAN to our two edge routers. The edge routers have an ingress L3 ACL permitting esp/ahp packets towards the PA loopback address. Removing the L3 ACL entirely also did not help.

Any advice/insight would be greatly appreciated!

Message was edited by: Aris Lamprianidis

Tags (2)
L7 Applicator

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Based on this information I believe the issue is on the other device.  Since you vpn shows decap of zero, this means no packets are coming out of the tunnel from the remote side.

If the PA were dropping or blocking by policy or inspection, you would see decap increment but nothing on the local host packet capture.

What do the IPSEC stats for the tunnel show on the remote node?

Steve Puluka BSEET - IP Architect - DQE Communications (Metro Ethernet/ISP)
ACE PanOS 6; ACE PanOS 7; ASE 3.0; PSE 7.0 Foundations & Associate in Platform; Cyber Security; Data Center
L5 Sessionator

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Could you paste the output of show session id <id number> for the traffic going from PA to Remote site.

L1 Bithead

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Hi Steven. That was my assumption on the first take. Then however I configured the two other peers (Cisco SOHO and Strongswan) and I see the same behavior..

Adding to my reply, the packet counter  for the SA from the Strongswan side *is* increasing (see line before last). This part of the output of "ipsec statusall":

Security Associations (1 up, 0 connecting):

      PA[1]: ESTABLISHED 20 minutes ago, 200.200.159.51[200.200.159.51]...100.100.139.9[100.100.139.9]

      PA[1]: IKEv1 SPIs: 46112c603ecc9b01_i 5c5827074640bb75_r*, pre-shared key reauthentication in 7 hours

      PA[1]: IKE proposal: AES_CBC_256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/MODP_1536

      PA{2}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: c9cb52d8_i d369e77c_o

      PA{2}:  AES_CBC_256/HMAC_SHA2_512_256, 68712 bytes_i (818 pkts, 220s ago), 70644 bytes_o (841 pkts, 1s ago), rekeying in 20 minutes

      PA{2}:    172.26.16.0/22 === 10.20.30.0/25

L1 Bithead

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Hi Pankaj, here you go (output redacted/sanitized):

"100.100.139.9" is the PA

"200.200.159.51" is the StrongSwan peer

10.20.30.0/25 is the post-NAT PA network

172.26.16.0/22 is the StrongSwan network

aris@PA(active)> show session all filter protocol 50

--------------------------------------------------------------------------------

ID          Application    State   Type Flag  Src[Sport]/Zone/Proto (translated IP[Port])

Vsys                                          Dst[Dport]/Zone (translated IP[Port])

--------------------------------------------------------------------------------

183171       ipsec-esp      ACTIVE  TUNN       100.100.139.9[51659]/vpn/50  (100.100.139.9[59260])

vsys1                                          200.200.159.51[21208]/vpn  (200.200.159.51[54121])

And this is the output of "ip xfrm policy" on the StrongSwan side:

# ip xfrm policy

src 10.20.30.0/25 dst 172.26.16.0/22

        dir fwd priority 2887

        tmpl src 100.100.139.9 dst 200.200.159.51

                proto esp reqid 1 mode tunnel

src 10.20.30.0/25 dst 172.26.16.0/22

        dir in priority 2887

        tmpl src 100.100.139.9 dst 200.200.159.51

                proto esp reqid 1 mode tunnel

src 172.26.16.0/22 dst 10.20.30.0/25

        dir out priority 2887

        tmpl src 200.200.159.51 dst 100.100.139.9

                proto esp reqid 1 mode tunnel

src 0.0.0.0/0 dst 0.0.0.0/0

        socket in priority 0

src 0.0.0.0/0 dst 0.0.0.0/0

        socket out priority 0

src 0.0.0.0/0 dst 0.0.0.0/0

        socket in priority 0

src 0.0.0.0/0 dst 0.0.0.0/0

        socket out priority 0

src ::/0 dst ::/0

        socket in priority 0

src ::/0 dst ::/0

        socket out priority 0

src ::/0 dst ::/0

        socket in priority 0

src ::/0 dst ::/0

        socket out priority 0

L1 Bithead

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Update: I've also tried setting up a s2s with a Fortigate device (not my configuration), and I still see the same issue.

Also note that the StrongSwan - Fortigate IPsec tunnel works OK.

L5 Sessionator

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Hello AMS-IX,

Initiate a ping from the host sitting behind the PA firewall to the host sitting behind the other side. There after check the session for that ping

show session all filter application ping source <> destination <>

show session id <>

L1 Bithead

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Hi Pankaj,

Here's the output:

aris@panix2(active)> show session all filter application ping source a.b.c.11

--------------------------------------------------------------------------------

ID          Application    State   Type Flag  Src[Sport]/Zone/Proto (translated IP[Port])

Vsys                                          Dst[Dport]/Zone (translated IP[Port])

--------------------------------------------------------------------------------

333341       ping           ACTIVE  FLOW  NS   a.b.c.11[59216]/int-servers/1  (10.20.30.11[59216])

vsys1                                          172.26.16.6[328]/vpn  (172.26.16.6[328])

aris@panix2(active)> show session id 333341

Session          333341

        c2s flow:

                source:      a.b.c.11 [int-servers]

                dst:         172.26.16.6

                proto:       1

                sport:       59216           dport:      39

                state:       INIT            type:       FLOW

                src user:    unknown

                dst user:    unknown

        s2c flow:

                source:      172.26.16.6 [vpn]

                dst:         10.20.30.11

                proto:       1

                sport:       39              dport:      59216

                state:       INIT            type:       FLOW

                src user:    unknown

                dst user:    unknown

        start time                           : Mon Aug 10 08:41:47 2015

        timeout                              : 6 sec

        total byte count(c2s)                : 102

        total byte count(s2c)                : 0

        layer7 packet count(c2s)             : 1

        layer7 packet count(s2c)             : 0

        vsys                                 : vsys1

        application                          : ping 

        rule                                 : IS-V-001

        session to be logged at end          : True

        session in session ager              : False

        session updated by HA peer           : False

        address/port translation             : source

        nat-rule                             : Internal Servers(vsys1)

        layer7 processing                    : enabled

        URL filtering enabled                : False

        session via syn-cookies              : False

        session terminated on host           : False

        session traverses tunnel             : True

        captive portal session               : False

        ingress interface                    : ae2.105

        egress interface                     : tunnel.2

        session QoS rule                     : N/A (class 4)

        tracker stage firewall               : Aged out

        end-reason                           : aged-out

L5 Sessionator

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Here traffic is not coming back to the PA firewall.

You can try doing PCAP on the PA firewall and then check where firewall is receiving the packets. Monitor>Packet  capture> set two filters for a.b.c.11 to 172.26.16.6 and 172.26.16.6 to a.b.c.11

then add all four pacp.

If you get drop pacp then it means firewall is dropping the packet if not then might be firewall is not receiving any packet.

L1 Bithead

Re: IPsec Site-to-Site VPN trouble (decap bytes 0)

Thanks for your reply, Pankaj.

I already in fact did that: I can see the ICMP echo request in the cluster, but no ICMP echo replies are ever seen.. I have already opened a support request with our integrator. I was hoping this whole thing to be a misconfiguration of some sort, but chances of that are getting slimmer..

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!