URL from URL filtering and SSL Decryption

[ovel]

Hi folks!

I have an andriod device that has a news app installed and also included into the SSL decrypted devices zone. The traffic from this app is apparently has a certificate pinning because it doesn't show anything once started. Obviously, the Palo signed CA cert is installed into this android. The trick is that the app starts working if I remove the "news" categroy from the Decryption rule. So that means that Palo FW somehow knows to which URL the app is trying to connect to. However, I tried to find the URL to whitelist the particular URL instead of the whole "news" category, but I couldn't. So how to identify this URL ? My logic is here: if Palo FW knows what URL category the traffic is falling to, how can i find this URL then ? Obviously there is nothing meaningful in the logs.

Same things happens with the Rome2Rio app (travel), ebay (shopping) and others. You can't see URLs they're trying to establish connections to anywhere, but once you remove the whole category from decryption list, everything starts working.

Thanks!

[Remo · Accepted solution]

Hi @ovel 

To find the URLs you need to do a packet capture on the firewall. In the packet capture search for tcp ssl client hello packets. In these packets there you will find the tls server name extension and there you can see the hostname that you need to add as decryption exclusion to make the app work again without adding too much as exclusion.

[BPry]

@ovel,

Are you actually logging all of the attempted URLs for that client, or simply the ones that were actually set to alert or higher. When you are attempting to debug something like this I always find it helpful to have a URL-Filtering profile with all categories set to alert so that I log all of the attempted URLs that the firewall is identifying, with the added benefit that you can see the URL categorization. 

Obviously what @Remo mentioned will work perfectly fine as well; I simply don't like taking packet captures during production hours and I'd rather not resort to digging through a PCAP for URLs unless I actually need to. 

[Remo]

@BPry 

When the decryption fails because of (like in this topic) pinned certs, there will be no URL log. Or did this change maybe in 9.0 or so? So far I am still using 8.1 on all production firewalls and did not have this situation so far in the lab.

[ovel]

Yes, correct. Palo doesn't show anything in the URL filtering logs when it falls into the cert pinning issue. But thank you for your advice, not an easy way to find the URL tho 🙂

Do you use any special filters in wireshark to extracet this URL from TLS hello ? The only way i found just looking at the packets content.