02-18-2016 05:33 AM
After migrating from an ASA to PA3020, users reported that web pages were not fully loading. The issue was seen on the ASA but rarely. The PA3020 has been showing this issue more often than not resulting in a work around being done on the webpage. The trouble appears to be tied to how the F5 and Palo communicate. With caching enabled, the problem becomes intermittent. With caching disabled, the site fails regularly. F5 support has referenced a bug within the Palo ASIC: https://www.reddit.com/r/networking/comments/3eshjz/update_tcp_zerowindow_issues/
In working with Palo support, they have stated this is not related as we are running 6.0.10 and the article is relating this to 6.1.5.
To troubleshoot the issue without impacting production, a test environment was created and the issue can be replicated. Prior to inserting the Palo into the mix, the sites did not experience issues. Packet captures on the link show multiple out of sync packets when the page hangs. This is a concern as the only traffic being generated in the test environment is when we are testing which is going through the websites.
Has anyone seen this issue or know of anything similiar which can point me into the right direction to resolve?
02-18-2016 09:23 AM
Issue seems to be asymmetric routing. Do pacps on firewall to confirm. Check the following documents:
02-18-2016 11:56 AM
Thanks for the links as this helped confirm no packets are being dropped due to asymmetric routing. The FW was placed into bypass mode globally after trying by zone and the problem can be reproduced. The site does not fail all the time which is what makes this rather odd. The page also seems to stop loading at the same point which our application team is looking into.
Can anyone think of something else which can be check? The F5 and site coding has not been ruled out as of yet either. Would be nice to eliminate something in this mix.
Fun stuff.
02-19-2016 12:07 AM
Set a packet capture filter for this traffic on all stages. Compare packets received on ingress interface of PA, firewalled and transmited on the egress interface. This way you can make sure if PA is dropping any packets. In drop stage you shouldn't see any packets relevant for the observed TCP session.
At the same time do packet capture on web server as well. Check if all packets arrive to web server. I don't know about F5 packet capture capabilities. But I would do similar capture as on PA on F5 as well if possible. Do them siultaneuosly to compare them for same session.
This should give you the answer what is happening (or at least where are packets dissapearing).
02-26-2016 05:55 AM
Got pulled away to build out a third environment and hopefully will get back on it soon.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!
