Upgrade to 5.0.14-h3 stopped traffic

[jambulo]

We just attempted to upgrade some 5020's to 5.0.14-h3(mainly to patch the evasion vulnerability) and quickly found that the upgrade broke traffic traversing the firewall.  During the short period of time it we were running on 5.0.14-h3, there were a whole lot of "incomplete" sessions for TCP and a lot of UDP sessions with zero packets received.

Does anyone else have experience with 5.0.14-h3?

[hshah]

I think you might have asymmetric traffic in network.

Is it among non-internet zone, if yes you might want to try following command.

show counter global | match syn

This will help us to determine potential asymmetric routing issue and fix. If values are high than apply following command.

> configure

# set deviceconfig setting session tcp-reject-non-syn no

# commit

Refer following document for more help.

SYN-ACK Issues with Asymmetric Routing

Regards,

Hardik Shah

[dreputi]

Hello Jambulo,

You can check if the configured interfaces are having proper arp entries on the PAN. Also, check the arp entries on the connected switches/routers. If the device is in HA mode, and if the connected devices didn't update the arp entries to the active device, you can try to clear the arp entries on those devices so that they learn the arp entries freshly. You can also try to run test gratuitous arp command to send out grat arps that will force connected devices to update their arp entries.

>test arp gratuitous ip <ip/netmask> interface <interface>

-Make sure the traffic is hitting the correct rules. For ex, if group-mapping is used in security rules, make sure that the users are properly identified so that they hit correct rules.

Regards,

Dileep

[hshah]

Hi Jabulo,

Following commands should help.

# set deviceconfig setting session tcp-reject-non-syn no |yes    <------- asymmetric routing

# set deviceconfig setting tcp asymmetric-path bypass | drop  <--------- asymmetric flow of packets

I am very positive its following issue.

6.0.5 h3 explanation

To verify same, provide us following output.

show counter global | match syn

Regards,

Hardik Shah

[jambulo]

Here is the output from  "show counter global | match syn"

flow_inter_cpu_nat_mismatch            22592        1 info      flow      pktproc   Inter-CPU NAT sync mismatch

ha_nat_policy_mismatch                104559        5 warn      ha        system    HA NAT session sync: policy mismatch

ha_nat_pool_mismatch                     814        0 warn      ha        system    HA NAT session sync: IP/port pool state mismatch

[hshah]

Hi Jambulo,

This doesnt look like a asymmetric routing issue. Please provide us traffic log snapshot. Make sure its enlarged view.

Regards,

Hardik Shah

[tshiv]

Hello jambulo,

I have come across this issue with 5.0.14-h3 software code. It is currently being investigated. It would helpful to us if you can open a support ticket and provide the necessary data. This issue needs to be investigated to find the root cause.

Thanks