We are facing currently this issue with a DC firewall. The following is the environment
EnduserPC-> DC Firewall (PAN) -> f5 Load Balancer-> Web Servers
All these days the users were able to login to the web services without any hassles. For the last 2 days, we found that the users are not able to access the application properly. f5 load balancer is showing lot of ssl handshake error.
We raised a support ticket with f5 and identified that the client is sending client hello and f5 is responding with server hello. However the server hello message is missing at the client side. The client keeps retransmitting packet and after about 10 sec the ssl session timeout happens.
While we did a packet capture at the firewall we noticed that the firewall indeed receive the server hello message. We are not sure why the firewall is dropping the server hello message. we have disabled all security inspection at the firewall including ssl decryption.
Firewall is running PAN OS 9.0.13 and we also tried upgrading this to 9.1.10. But still the issue persists.
Can someone guide us what could be the issue.
When you say the firewall is dropping this, did you see the server hello in the 'drop' stage capture? Or did you see it in the receive stage capture, but not the transmit?
If you see it in the drop stage, you can usually get more info on the drop reason by checking the global counters:
- When your pcap filter is set and enabled, log onto the CLI
- Run the command 'show counter global filter packet-filter yes delta yes'
- Test the connection, allow it to fail
- Run the command 'show counter global filter packet-filter yes delta yes' - this time check for any 'drop' counters
It's important that you also check the pcap at the 'transmit' stage - if you see the packet in the transmit stage, it is not dropped by the firewall. 99% of the time when I see and issue with server hello dropping somewhere, it's because of MTU as it's the first part of the handshake which will send a full sized packet.
Basically, make sure you pcap at receive, transmit and drop. Receive only is not enough for a diagnosis.
Thank you for the response.
All of a sudden without any change to configuration or environment change, the app started working through the firewall for the last 8 hours. The service has been down for more than 50 hours, Since it is a long week end we will not be able to observe this in the next 3 days. We will check pcap at all the stages if the problem repeats again and then revert back here.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!