topic Re: SSL Decryption in different countries? in General Topics

SSL Decryption in different countries?

Rievax — Thu, 10 Oct 2019 19:40:07 GMT

Hello All,

Starting to deploy 100+ firewalls worldwide. Have configured SSL decryption for General Browsing rule.

A template has been configured in Panorama, so they all have the exact same setup.

North America and Europe locations I tested are OK. Tried a Brazil office yesterday and if decryption is enabled, for very basic sites like UPS and Fedex, it becomes super slow - sometimes does not even finish loading the page at all. As soon as I disable decryption in this Brazilian branch, all seems to be working fine. Looked into the logs and there were many failures (decrypt-error) for the Session-End Reason. I do not have similar logs in other branch offices in North America and Europe.

What would be your input on that?

Thanks!

Re: SSL Decryption in different countries?

nagarjuna.b — Fri, 11 Oct 2019 04:14:23 GMT

Hi @Rievax

Decrypt-error tells us the ciphers used by the web server are not supported/matched with the ciphers enabled on decryption profile. Can you verify that.

Regards, Nagarjuna

Re: SSL Decryption in different countries?

Rievax — Fri, 11 Oct 2019 13:50:01 GMT

Hello @nagarjuna.b

Thanks for the answer.

This morning, I did re-enable decryption again for a specific test workstation but had none of those Decrypt-errors I had yesterday at implementation time... No clue from where is was coming. To answer more specifically your question, the Decryption profile is quite open (like the Default) and the web site goes with TLS 1.2, ECDHE_RSA with P-256, and AES_256_GCM... which is exactly what PA generates when decryption is enabled. That being said, the loading page delays are still there.

Focusing on UPS.COM web site, when decryption is enabled in this particular location (big city in Brazil), it takes 5 seconds to load the root site (/) and 5 to 7 minutes to load some JPGs and GIF files (numbers are coming from Developer Tools in Firefox / Chrome) - making the page virtually hang. Disabling decryption makes this site display in about a second. I did not see any kind of related information in Data Filtering / Wildfire Submissions logs. In other Office Locations around the globe that I tested, I did not have these delays with decryption enabled.

Interestingly, many other sites are just working OK but also notice delays in loading (or non-loading) images. I though that could be an interesting point.

Any help / clues are appreciated. Thanks!

Re: SSL Decryption in different countries?

OtakarKlier — Fri, 11 Oct 2019 20:16:56 GMT

Hello,

Have you looked at the logs to make sure nothing is getting blocked? OR taken any pcaps to see if there are a lot of retransmits or other issues?

Just some thoughts.

Re: SSL Decryption in different countries?

Rievax — Fri, 18 Oct 2019 19:28:59 GMT

Hello,

Sorry for the late answer as had to test in other places to get a base reference.

So I went with the suggestion to capture packets (at the PA Firewall transmit stage) and noticed that in Brazil, I have lots of fragmentation happening:

Now, I will try to reach the ISP but it seems from the Cisco router suggested configuration that the MTU is 1492. I confirmed that by using ping packets of 1464 (+28 of overhead) - higher are failing:

Our WAN tech tried to manually set 1500 but the end-result is the same. We will have to talk to the ISP now.

That being said, could this be the reason why decryption is failing? Some odd issues when the PA tries to re-assemble the packets and scan the images? Quite new to PA and decryption, so I definitely don't know that answer - yet.

Again, thanks for any comments.

Regards.