Issue with traffic on specific proxy id

[yshaikh]

We have VPN between Palo Alto and Cisco FMC/FTD.

There is user and server traffic on VPN. VPN status is stable. I don't have any user complaining about disconnection.

But I am seeing disconnection on specific proxyid. All of sudden I am getting ICMP request time out on working connection.

Facing request time out when ping is from Server which is behind Palo Alto.

To make connection up, either I need to generate Interesting traffic from FMC or ping from Server which is behind FMC. This restore request time out issue from the Server which is behind Palo Alto.

I don't undertstand what could be reason for behind this specific connection.

yshaikhadmin@SPDORC-FW02(active)> show vpn flow tunnel-id 358

Request time out started, checked vpn flow:

yshaikhadmin@SPDORC-FW02(active)> show vpn flow tunnel-id 358

tunnel Orbit:test-mig-1
id: 358
type: IPSec
gateway id: 9
local ip: x
peer ip: x
inner interface: tunnel.17
outer interface: ethernet1/1
state: active
session: 444419
tunnel mtu: 1428
lifetime remain: 2475 sec
lifesize remain: 4607944 kb
latest rekey: 1125 seconds ago
monitor: off
monitor packets seen: 0
monitor packets reply:0
en/decap context: 5679
local spi: 9F518E95
remote spi: 8EBEAD75
key type: auto key
protocol: ESP
auth algorithm: SHA1
enc algorithm: AES256
proxy-id:
local ip: x
remote ip: x
protocol: 0
local port: 0
remote port: 0
anti replay check: no
copy tos: no
authentication errors: 0
decryption errors: 0
inner packet warnings: 0
replay packets: 0
packets received
when lifetime expired:0
when lifesize expired:0
sending sequence: 645
receive sequence: 0
encap packets: 58580
decap packets: 14429
encap bytes: 5230576
decap bytes: 1274216
key acquire requests: 1
owner state: 0
owner cpuid: s1dp0
ownership: 1

when I started ping from server behind cisco FMC then ping restored. I can see that rekeying happens.

tunnel Orbit:test-mig-1
id: 358
type: IPSec
gateway id: 9
local ip: x
peer ip: x
inner interface: tunnel.17
outer interface: ethernet1/1
state: active
session: 197328
tunnel mtu: 1428
lifetime remain: 3594 sec
lifesize remain: 4607999 kb
latest rekey: 6 seconds ago
monitor: off
monitor packets seen: 0
monitor packets reply:0
en/decap context: 391
local spi: 8D78AF05
remote spi: 90040096
key type: auto key
protocol: ESP
auth algorithm: SHA1
enc algorithm: AES256
proxy-id:
local ip: x
remote ip: x
protocol: 0
local port: 0
remote port: 0
anti replay check: no
copy tos: no
authentication errors: 0
decryption errors: 0
inner packet warnings: 0
replay packets: 0
packets received
when lifetime expired:0
when lifesize expired:0
sending sequence: 4
receive sequence: 0
encap packets: 58593
decap packets: 14433
encap bytes: 5231720
decap bytes: 1274568
key acquire requests: 1
owner state: 0
owner cpuid: s1dp0
ownership: 1

yshaikhadmin@SPDORC-FW02(active)>

I think after rekeying process, some how Palo Alto not able to keep this connection alive, not sure why

[BPry]

@yshaikh,

While it's odd that your ICMP request isn't keeping the connection alive by itself, since that should be generating interesting traffic to keep the tunnel up on the Cisco side, it sounds like Cisco is really who you should be reaching out to in this scenario. 

[yshaikh]

if I do icmp from server behind PA firewall nothing happens.

if I do icmp from server behind Cisco firewall, ping becomes ok

I discussed with Cisco also, they are saying you need to check with Palo Alto why Palo Alto doesn't send anything on VPN for that specific connection, why connection gets alive when do ping from Cisco Server 

Cisco:

> Found no debug logs for the specific SA until we start it manually

> Looks like the remote end is unable to start an SA for the particular traffic

[BPry]

@yshaikh,

The connection is re-established when you ping from the Cisco side because Cisco will bring their side back up when it has interested traffic. By default, if it hasn't seen any interesting traffic it will bring that tunnel offline until such time that interesting traffic is passed. This isn't a PAN behavior, you could have absolutely no interesting traffic traverse that tunnel and PAN doesn't care and will keep the tunnel up.

Do you have the PAN side monitoring the connection and actively sending ICMP traffic down that SA at regular intervals, or are you only checking once something is reported as non-reachable? Do you see regular traffic crossing the firewall for that SA or is the traffic minimal? I'd be very surprised to see this be an issue on the PAN side of things to be honest; this sounds like the Cisco side is collapsing the SA due to non-interesting traffic.

[yshaikh]

@BPry Thanks for your responses and time, really appreciate it.

I don't have issue with other proxy id on the same VPN tunnel. Ping works either way. Its just this specific one.

There is no disconnection on other proxy ids

[pramodl]

HI!

I have the same issue..

have you found a way around for this ?

Regards

Pramod