11-10-2020 09:34 AM
Recently we changed to 9.0 code. We are running decryption on our firewalls. I've seen some very slow google searches recently, and a few errors when searching all while using chrome. Eventually the page will load the search if I wait long enough. It's almost like chrome is failing the search at first, and then succeeding? Wondering if this has something to do with http2 decryption?
11-10-2020 01:28 PM
See if you can set chrome in developer mode (or install a.http plugin) to track how many queries are being launched when running a.search. since google is all about tracking etc you're probably launching a handful of sessions that all need to be decrypted and some may wait for others to complete so google can optimize your 'ad' spread
You could try either blocking some ad categories or disabling decryption for them
11-10-2020 02:46 PM
This is interesting. is there a plugin you suggest that is best for this?
11-10-2020 03:36 PM - edited 11-10-2020 03:38 PM
I used to use 'live http headers' bit that's been infested with malware some time ago
The built-in dev tool is pretty good though: hit shift+ctrl+J or option+command+J and then hit the "network" tab. Then see what happens if you do a Google search (mine currently shows 60 requests)
This will also help you visualize where the delay is (are all queries slow or are there 1/few that are slowing everything down)
11-10-2020 06:32 PM - edited 11-10-2020 06:35 PM
Just random ones, but it's the majority. Seems you can replicate by closing the browser and relaunching it and googling something. Even went as far as allowing web advertisements category for google-base, to see if that would help. Same thing. Edge surfs just fine, even on google. Just chrome. Anything to do with this?
11-11-2020 04:35 PM - edited 11-11-2020 04:37 PM
it's interesting, as I see really long "Stalled" - some up to 50 -60 seconds. All of these searches have the "h2" protocol involved. If I strip alpn, all returns to normal and I can't replicate it. All of the issues starts with a fresh window of chrome, and the first search you enter. Below of an h2 transaction for a simple google search:
12-03-2020 04:43 AM
Did you get anywhere with this? I've got the same situation at one location, 9.1.5, so probably not firmware related. I found if you disable TLS inspection for a test user/computer the issue goes away, so probably something to do with inspection/decoders. Going to do some more investigation tomorrow. I did some PCAPs and it's very strange, there's just a massive delay in the response, but nothing actually dropped or blocked by the firewall.
12-03-2020 08:51 AM
Just a shot in the dark here. If I remember right, Chrome uses QUIC. Maybe try blocking it and see what happens. I block it on my PA so I can decrypt the traffic. https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/decryption/define-traffic-to-decrypt.html
Just another I.T. Guy
12-03-2020 08:54 AM
very true, and a good shout. I'm 99% certain it's blocked on this deployment but could be slipping through the net or getting borked in some way ! I'll check that too. thank you!
12-03-2020 10:03 AM
Hello again, managed to cram an hour of tinkering into the end of the day.
Not quic. it's blocked at the firewall with an app rule, i also disabled quic in the chrome browser to rule it out totally.
However, did make some progress, looking like the http2 decoder on the firewall. We had some similar issues with totally different sites ages ago when 9.0 first came out. (feels like a lifetime ago now)
Created an inspection profile with the strip ALPM option checked, this functionally disabled http2 and forces it down to plain old http.
Have put in a decrypt rule with URL a custom URL category only containing www.google.com and this option enabled.
seems to have fixed it. I wonder if the OP has the same issue, would be cool to hear!
Will see how it goes, also going to do some rummaging around and see if it's an issue at other locations and the users just haven't reported anything. Certainly dont see it where I am, but we're PANOS 10.0.2 pilot site!
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!