06-29-2021 08:18 AM
Im involved in a project to migrate away from old asa firewalls to a palo solution.
The process has gone well but myself and peers are stumped with an odd issue and looking for troubleshooting advice.
We have a number of https hosts in a dmz, nat'ed to be available to the public internet. systems from all over the world can access these https hosts fine. That is.... except for a small group of individuals all using a particular local ISP. The trouble is, this issue did not exist with the previous ASA firewalls.
The security policy is simple, allow tcp 443 in. People with this issue will intermittently have difficultly fully loading an https site behind our palos. Pages will partially load or time out. Chrome debug logs from the end users perspective show timeouts on connection attempts.
If a user with the issue retries enough they will eventually establish a session and have no troubles. The https hosts in our DMZ show connection attempts from users on this ISP and from their perspective traffic seems to just stop.
No drops or blocks appear in threat or security logs on the palo and we dont see this behavior with any other people from other ISPs.
The ISP has insisted they are not mangling traffic and from what Ive seen I agree but the problem persists. Ordinarily Id write this off as an ISP doing something odd but the problem didnt exist when our prior ASA firewalls were in place.
The issue feels like a content inspection or ssl decryption issue but we've confirmed thats not in play. We've also confirmed any outside IDS systems are not mangling or dropping traffic. We dont see any asyc routing in play so it does not appear the palo is loosing state and dropping traffic. Everything from what we see appears the same as it was but still we continue to have issues with people using this one particular ISP
Myself and peers are looking for further troubleshooting insights or advice.
Any advice would be greatly appreciated.
06-30-2021 08:17 AM
LOL thats pretty good.
Thanks for the ACK advice! Sounds plausible. Ill check it out.
06-30-2021 08:20 AM
Use TAC, that is what they are there for. Take advantage of the service you paid for 🙂
06-30-2021 08:40 AM
I peeked at our palos at each datacenter one has the setting turn on, the other has it turned off.
show running tcp state
Allow Challenge Ack : yes
show running tcp state
Allow Challenge Ack : no
Back to the drawing board unfortunately... that challenge ack article really seemed promising
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!