Active/Active & IPSec Trouble

Reply
Highlighted
L2 Linker

Active/Active & IPSec Trouble

After implementing HA Active/Active, we left S2S VPN tunnels alone.  Ultimately no changes to IKE Gateways.  The S2S terminate to a /30 address that is statically routed from the ISP to ethernet1/12 on the active-primary.  Tunnel interfaces and their routes are identical on both primary and secondary, IKE Gateways are NOT synced as this is just an IP assigned to the 1/12 interface on the primary side.  OSPF on the internal LAN side, HOWEVER, supplied static routes for the remote sites to only route to primary device.  Secondary should never be session owner.

Session Owner = First Packet

Session Setup = ip-modulo

Phase 1 and Phase 2 seem to come up fine.  Communication is established and devices can talk fine.  However, the key lifetime is set for 2 hours.  Throughout the day, the remote side will stop responding at key expiration.  It appears that a new negotiation is started and completed, however for the two hour key time the VPN tunnels will not pass traffic.  This doesn't happen every time it negotiates, and I can seem to locate a pattern.  The best lead that I have right now is that the return traffic from the remote side is being discarded due to the flow being in a discard state.  I can also see where the 'decap packets' in the vpn flow will not increment.  I've looked through the ike debug log and don't really see anything going on there during this state of limbo.  Phase 1 appears to be up, phase 2 SAs seem to be up, but for some reason the firewall is rejecting these return packets and I cannot determine why.  If can clear phase 1, and the flows and it will pick right back up.  Other option is to just wait for Phase 1 to timeout and re-key.  These devices were working fine prior to HA to the best of my knowledge.

Any help would be greatly appreciated!

== Apr 07 09:33:26 ==

Packet received at ha_ingress stage

Packet info: len 1532 port 49 interface 49 vsys 1

  wqe index 228383 packet 0x0x8000000416f718e0

Packet decoded dump:

L2:     00:1b:17:3c:21:31->00:1b:17:c8:6f:31, type 0x7261

L3 binary dump: 40 bytes

00000000: 80 02 00 1b 00 1b 17 3c  21 1b 00 16 4d 6e a6 8c    .......< !...Mn..

00000010: 08 00 45 00 05 dc c8 d9  40 00 36 06 ff 43 c7 1b    ..E..... @.6..C..

00000020: 4e b8 41 29 20 02 00 50                             N.A) ..P

...skipping...

IP:     166.xxx.xxx.xxx->71.xxx.xxx.xxx, protocol 50

        version 4, ihl 5, tos 0x00, len 120,

        id 663, frag_off 0x0000, ttl 49, checksum 57440

L4 binary dump: 16 bytes

00000000: f3 46 fb c0 00 00 01 3c  19 ac 55 94 f5 04 3f a5    .F.....< ..U...?.

Flow lookup, key word0 0xf346fbc000023200 word1 0

Flow 211818 found, state 3

Packet dropped, flow in discard state

Session          105909

        c2s flow:

                source:      166.xxx.xxx.xxx [untrust]

                dst:         71.2.234.230

                proto:       50

                sport:       62278           dport:      64448

                state:       DISCARD         type:       FLOW

                src user:    unknown

                dst user:    unknown

        s2c flow:

                source:      71.xxx.xxx.xxx [untrust]

                dst:         166.142.205.229

                proto:       50

                sport:       64448           dport:      62278

                state:       DISCARD         type:       FLOW

                src user:    unknown

                dst user:    unknown

        start time                    : Mon Apr  7 09:28:39 2014

        timeout                       : 60 sec

        time to live                  : 58 sec

        total byte count(c2s)         : 103868

        total byte count(s2c)         : 0

        layer7 packet count(c2s)      : 722

        layer7 packet count(s2c)      : 0

        vsys                          : vsys1

        application                   : ipsec-esp

        rule                          : ALLOW_IN_VPN

        session to be logged at end   : True

        session in session ager       : True

        session synced from HA peer   : False

        session owned by local HA A/A : True

        layer7 processing             : completed

        URL filtering enabled         : True

        URL category                  : any

        session via syn-cookies       : False

        session terminated on host    : True

        session traverses tunnel      : False

        captive portal session        : False

        ingress interface             : ethernet1/12

        egress interface              : ethernet1/12

        session QoS rule              : N/A (class 4)

        session tracker stage deny    : host service

        session tracker stage l7proc  : ctd app has no decoder

Community Team Member

Re: Active/Active & IPSec Trouble

What version are you at now?

I have seen a similar A/A issue with vpn's that break in the middle of the key-renegotiate with 5.0.9.

There are a couple of solutions..

1. Upgrade to 6.0.1, and this issue should go away.

2. Test with 1 member shut off/disabled.  IF that works, then you can try to Configure Active/Passive, but not sure if that is an option for you.

I hope this helps.

Stay Secure,
Joe
End of line
L2 Linker

Re: Active/Active & IPSec Trouble

Thank you jdelio, I think you nailed it!

We are at 5.0.11 and just upgraded to that a week prior to implementing HA.

I've had a support case open since starting with Active/Active.  They responded today that it is most likely they bug you are talking about but I'm reluctant to upgrade to 6.0.1 however.

From the 6.0.1 Release Notes - Resolved Issues

• 60201—In a HA Active/Active setup, an IPSec key renegotiation timing issue caused the new IPSec session to be set to DISCARD until the next key renegotiation. This caused traffic loss until the next tunnel key renegotiation.

L2 Linker

Re: Active/Active & IPSec Trouble

Have you had any issues with 6.0.1?

Community Team Member

Re: Active/Active & IPSec Trouble

Honestly, there have been some issues here and there, impossible not to have any.. but they were either resolved with rebooting the unit a couple of times to clear a stuck state or ensuring that there is a subnet mask on the IPSEC/VPN tunnel interfaces.  Other than that, I would recommend looking through the "known issues" in the release notes.

Glad I could help.

Stay Secure,
Joe
End of line
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!