How to Implement and Test SSL Decryption

by nrice on ‎04-17-2010 03:39 PM - edited on ‎12-02-2016 08:20 AM by (224,263 Views)

Overview

PAN-OS can decrypt and inspect inbound and outbound SSL connections going through the Palo Alto Networks firewall. SSL decryption can occur on interfaces in virtual wire, Layer 2 or Layer 3 mode by using the SSL rulebase to configure which traffic to decrypt. In particular, decryption can be based upon URL categories and source user and source/target addresses. Once traffic is decrypted, tunneled applications can be detected and controlled, and the decrypted data can be inspected for threats, URL filtering, file blocking, or data filtering. Decrypted traffic is never sent off the device.

 

Inbound SSL Decryption

In the case of inbound traffic to an internal Web Server or device, the administrator imports a copy of the protected server’s certificate and key. When the SSL server certificate is loaded on the firewall, and a SSL decryption policy is configured for the inbound traffic, the device then decrypts and reads the traffic as it forwards it. No changes are made to the packet data, and the secure channel is from the client system to the internal server. The firewall can then detect malicious content and control applications running over this secure channel.

 

Outbound SSL Decryption (SSL Forward Proxy)

In this case, the firewall proxies outbound SSL connections by intercepting outbound SSL requests and generating a certificate on the fly for the site the user wants to visit. The validity date on the PA-generated certificate is taken from the validity date on the real server certificate.

The issuing authority of the PA-generated certificate is the Palo Alto Networks device. If the firewall’s certificate is not part of an existing hierarchy, or is not added to a client’s browser cache, then the client receives a warning when browsing to a secure site. If the real server certificate has been issued by an authority not trusted by the Palo Alto Networks firewall, then the decryption certificate is using a second “untrusted” Certificate Authority (CA) key to ensure the user is warned of any subsequent man-in-the-middle attacks.

 

To configure SSL decryption:

  1. Configure the firewall to handle traffic and place it in the network.
  2. Make sure the proper Certificate Authority (CA) is on the firewall.
  3. Configure SSL decryption rules.
  4. Enable SSL decryption notification page (optional).
  5. Commit changes and test decryption.

 

Steps

1. Configure the firewall to handle traffic and place it in the network

Make sure the Palo Alto Networks firewall is already configured with working Interfaces (Virtual Wire, Layer 2 or Layer 3), Zones, Security Policy and already passing traffic.

 

2. Load or Generate a CA certificate on the Palo Alto Networks firewall

A Certificate Authority (CA) is required to decrypt traffic properly by generating SSL certificates on the fly. Create a self-signed CA on the firewall or import a Subordinate CA (from your own PKI infrastructure). Select "Forward Trust Certificate" and "Forward Untrust Certificate" on one or more certificates to enable the firewall to decrypt traffic.

Note: Because SSL Certificate providers like Entrust, Verisign, Digicert, and GoDaddy do not sell CAs, they are not supported in SSL Decryption.

 

From the firewall GUI, go to Device > Certificates. Load or generate a certificate for either inbound inspection or outbound (forward proxy) inspection.

 

Generating a Self-Signed Certificate

Using a Self-Signed Certificate is recommended. For information on generating a Self-Signed Certificate, please see:

How to Generate a New Self-Signed SSL Certificate

 

Generating and Importing a Certificate from Microsoft Certificate Server

  1. On the Microsoft Certificate Server for your organization, request an advanced certificate using certificate template “subordinate CA”. Download the cert.
  2. After downloading, export the certificate from the local certificate store. In IE, access the Internet Options dialog, select the Content tab, then click the Certificates button. The new certificate can be exported from the Personal certificates store. Select “Certificate Export Wizard”, export the private key, then select the format. Enter a passphrase and a file name and location for the resulting file. The certificate will be in a PFX format (PKCS #12).
  3. To extract the certificate, use this openSSL[4] command:
    openssl pkcs12 –in pfxfilename.pfx –out cert.pem –nokeys
  4. To extract the key, use this openSSL command:
    openssl pkcs12 –in pfxfilename.pfx –out keyfile.pem -nocerts
  5. Import the cert.pem file and keyfile.pem file into the Palo Alto Networks firewall on the Device tab > Certificates screen.
  6. In the case of a High Availability (HA) Pair, also load these files into the second Palo Alto Networks firewall, or copy the certificate and key via the High Availability widget on the dashboard.

 

The "Forward Trust" and "Forward Untrust" certificates:

cert-info-ssl-decrypt.png

 

 

Note: If using a self-signed CA, export the public CA Certificate from the firewall and install the certificate as a Trusted Root CA on each machine's browser to avoid Untrusted Certificate error messages inside your browser. Network administrators usually use GPO to push out this certificate to each workstation.

 

Examples of browser errors if the self-signed CA Certificate is not trusted:

 

Firefox untrusted CA error:

ssl-decrypt1.png

 

Chrome untrusted CA error:

ssl-decrypt2.png

 

Internet Explorer untrusted CA error:

ssl-decrypt3.png

 

3. Configure SSL Decryption Rules

The network administrator determines what needs to be decrypted. A few suggestions for configuring SSL decryption rules:

  • Implement rules in a phased approach. Start with specific rules for decryption, and monitor the typical number of SSL connections being decrypted by the device.
  • Avoid decrypting the following URL categories, as users may consider this an invasion of privacy:
    • Financial services
    • Health and medicine
  • Do not decrypt applications where the server requires client-side certificates (for identification).
    • You can either block or allow connections requiring client authentication via the decryption profile feature introduced in PAN-OS 5.0.

 

An example of an outbound rulebase following suggestions for decryption.

2015-03-10 12_45_40-77B-PA-VM-100.png

4. Enable SSL Decryption Notification web page (optional)

  • The user can be notified that their SSL connection will be decrypted using the response page found on the Device tab > Response Pages screen. Click "Disabled," check the "Enable SSL Opt-out Page" option and click OK.

Untitled.jpg

 

The default SSL Opt-out page page can be exported, edited via an html editor, and imported to provide company-specific information:

1.png

5. Test Outbound Decryption

To test outbound decryption:

  • Make sure that in the outbound policy, the action is to alert for any viruses found. Also enable packet capture on that anti-virus security profile. Commit any changes made.
  • On a PC internal to the firewall, go to www.eicar.org. In the top right corner:

Untitled.jpg

  • Click “Download anti-malware testfile."
  • In the screen that appears, scroll to the bottom.
  • Download the eicar test virus using http. Any of the these four files will be detected.

Untitled.jpg

  • Go to the Monitor tab > Threat log, and look for the log message that detects the eicar file.

ssl-eicar-log1.png

  • Click the green arrow in the column on the left to view the captured packets.
    ssl-eicar-log2.png

ssl-eicar-cap1.png

  • Click the magnifying class in the far left column to see the log detail.

ssl-eicar-log3.png

  • Scroll to the bottom, and look for the field “Decrypted.” The session was not decrypted:

ssl-eicar-log-detail-nodecryp1.png

 

  • Go back to the www.eicar.org downloads page. This time use SSL enabled protocol HTTPS to download the test virus.

Untitled.jpg

  • Examine the Threat logs. The virus should have been detected, since the SSL connection was decrypted. A log message that shows Eicar was detected in web browsing on port 443 will be visible.
    ssl-eicar-log4.png

  • View the packet capture (optional) by clicking the green arrow.

ssl-eicar-cap2.png

  • To the left of that log entry, click the magnifying class. Scroll to the bottom. Under Flags, check to see that “Decrypted” is checked:

ssl-eicar-log-detail-decryp1.png

 

The virus was successfully detected in an SSL-encrypted session.

 

To test the “no-decrypt” rule, first determine what URLs fall into financial services, shopping, or health and medicine categories. For BrightCloud, go to http://www.brightcloud.com/testasite.aspx. For PAN-DB, use Palo Alto Networks URL Filtering - Test A Site , and enter a URL to see what the category is.

Once web sites that are classified into categories that will not be decrypted are found, use a browser to go to those sites using https. There should be no certificate error when going to those sites. The web pages will be displayed properly. Traffic logs will show the sessions where application SSL traverses port 443, as expected.

 

To Test Inbound Decryption:

  • Examine the traffic logs dated before enabling SSL for inbound decryption on the firewall. Look at traffic targeted for the internal servers. In those logs, the application detected should be “ssl" going over port 443.
  • From a machine outside of the network, connect via SSL to a server in the DMZ. There will be no certificate errors, as the connection is not being proxied, just inspected.
  • Examine the logs for this inbound connection. The applications will not be “ssl" but the actual applications found inside the SSL tunnel. Click the magnifying glass icon in those log entries to confirm decrypted connections.

decrypted.png

 

Helpful CLI Commands:

To see how many existing SSL decryption sessions are going through the device:

> debug dataplane pool statistics | match Proxy

 

Output from a PA-2050, where the first command shows 1024 available sessions, and the output of the second command shows five SSL sessions being decrypted (1024–1019=5):

admin@test> debug dataplane pool statistics | match Proxy

[18] Proxy session            :    1019/1024    0x7f00723f1ee0

 

To see the active sessions that have been decrypted:

> show session all filter ssl-decrypt yes state active

Maximum number of concurrent SSL decrypted sessions in PAN-OS 4.1, 5.0, 6.0 and 6.1 (both directions combined):

Hardware SSL Decypted Session Limit
VM-100 1,024 sessions

VM-200

1,024 sessions
VM-300 1,024 sessions
PA-200 1,024 sessions
PA-500 1,024 sessions
PA-2020 1,024 sessions
PA-2050 1,024 sessions
PA-3020 7,936 sessions

PA-3050

15,360 sessions
PA-3060 15,360 sessions
PA-4020 7,936 sessions
PA-4050 23,808 sessions
PA-4060 23,808 sessions
PA-5020 15,872 sessions
PA-5050 47,616 sessions
PA-5060 90,112 sessions
PA-7000-20G-NPC 131,072 sessions
PA-7050 786,432 sessions

 

If the limit is reached, all new SSL sessions go through as undecrypted SSL. To drop any new SSL sessions beyond the session limit of the device:

> set deviceconfig setting ssl-decrypt deny-setup-failure yes

To check if there are any sessions hitting the limit of the device:

> show counter global name proxy_flow_alloc_failure

To view the SSL decryption certificate:

> show system setting ssl-decrypt certificate
Certificates for Global

SSL Decryption CERT
global trusted
ssl-decryption x509 certificate
version 2
cert algorithm 4
valid 150310210236Z -- 210522210236Z
cert pki 1
subject: 172.16.77.1
issuer: 172.16.77.1
serial number(9)
00 b6 96 7e c9 99 1f a8  f7                      ...~.... .
rsa key size 2048 siglen 2048
basic constraints extension CA 1

global untrusted
ssl-decryption x509 certificate
version 2
cert algorithm 4
valid 150310210236Z -- 210522210236Z
cert pki 1
subject: 172.16.77.1
issuer: 172.16.77.1
serial number(9)
00 b6 96 7e c9 99 1f a8  f7                      ...~.... .
rsa key size 2048 siglen 2048
basic constraints extension CA 1

 

To view SSL decryption settings:

> show system setting ssl-decrypt setting

vsys                          : vsys1
Forward Proxy Ready          : yes
Inbound Proxy Ready          : no
Disable ssl                  : no
Disable ssl-decrypt          : no
Notify user                  : no
Proxy for URL                : no
Wait for URL                  : no
Block revoked Cert            : yes
Block timeout Cert            : no
Block unknown Cert            : no
Cert Status Query Timeout    : 5
URL Category Query Timeout    : 5
Fwd proxy server cert's key size: 0
Use Cert Cache                : yes
Verify CRL                    : no
Verify OCSP                  : no
CRL Status receive Timeout    : 5
OCSP Status receive Timeout  : 5
Block unknown Cert            : no

For a list of resources about SSL Decryption, please refer to the following:

SSL Decryption Quick Reference - Resources

 

For more information on supported Cipher Suites for SSL Decryption, please refer to the following:

SSL Decryption Not Working Due to Unsupported Cipher Suites

Limitations and Recommendations While Implementing SSL Decryption

How to Identify Root Cause for SSL Decryption Failure Issues

 

Note: If anything else needs to be added to this document, please comment below.

 

owner: jdelio

Comments
by sm0lds
on ‎08-05-2010 06:40 AM

The suggestions in step 3 are a good cautionary note. Managing SSL decryption imposes an administrative responsibility. Just because you can, doesn't necessarily mean you should.

by migration
on ‎02-17-2011 10:32 AM

Hi Teresa,

would you mind clarifying me some points?

1. What does exactely mean this phrase?

If the real server certificate has been issued by an authority not trusted by the PA firewall, then the decryption certificate will be issued using a second “untrusted” CA key.  This is to insure that the user will be warned if there are subsequent man-in-the-middle attacks occurring.

Does it mean that I have to import on PA device (under Trusted CA certificates) all the Certificates Authority stored on the user's browser? It could be hundreds...

What are the trusted authority by the PAN device? For example: I have to import on PAN device the VeriSigh/Thawte CA certificate?

2. What does mean Client CA certificate? When and how I have to use it?

Unfortunatelly there's no clear documentaion about this.

I underestood that:

- I can generate self signed certificate if I don't have any sub CA certificate

- I have to Import under SSL Forward Proxy the Subordinate CA's certificate if I don't want the browser warning

- I didn't understand how and when I can/must use: Trusted CA Certificate and Client CA certificate.

I hope you or someone can help me, because it's VERY important during a POC.

Thanks in advance

by AndyPandy
on ‎09-06-2012 07:21 AM

Hi many thanks for the article very helpful, I am running  4.1.6, but I do not get the option to as in step 2 to generate a self signed certificate.

Am i missing something here or has the version I have not have this option ? or is it presented in a different way ?

Any help is appreciated

Tom

by EdwinD
on ‎09-06-2012 03:32 PM

If you choose to not decrypt the category of Shopping, then Amazon.com will fall into this shopping category and not be decrypted.  Amazon.com has things such as online video similar to Netflix and Hulu, and they have cloud files.  Someone can store their entire MP3 library in Amazon's Cloud Files and play them back with Amazon Cloud Player.    While the Palo Alto has an APP-ID for these apps, it doesn't work unless you decrypt the traffic.

by Shayan
on ‎11-19-2013 07:38 PM

Hi,

Very good article. How do I check whether the inbound SSL decryption works. Here is my scenario,

I am doing a NAT rule for EXchange OWA (Public to Private) and want to do inbound SSL decryption for the traffic. I have setup a decrypt rule for the public IP not the NAT (private). I want to verify whether this is working. What would be the process of doing this?

Shayan

by oschuler
on ‎07-09-2014 12:41 AM

Thank you for the article. Could you please tell us the max. number of sessions the SSL proxy can decrypt on the 3000 series devices? Looks like we hit the roof with our 2050 already :-(...

by pulukas
on ‎07-09-2014 02:57 AM

The SSL decryption limits are by bandwidth in processed sessions not number of sessions.  The current estimates are not published but you can get them from your PA sales engineer.

You can buy some time by looking at what sites are being processed and adding those to your exclusion.

by oschuler
on ‎07-09-2014 07:57 AM

Thank you Steven. Are you sure that the limit is not based on the amount of SSL proxied concurrent sessions? The article above talks of sessions and the output on the CLI (debug dataplane pool statistics) also shows

[16] Proxy session: 8/1024

Also, if I use the "Compare Products" feature on the PA homepage (see link below) they also describe this as a limit of max. concurrent sessions (chapter "SSL Decryption"):

https://www.paloaltonetworks.com/products/product-comparison.html?chosen=pa-3050,pa-2050

by pulukas
on ‎07-09-2014 04:56 PM

You are probably right looking at the documents that there is also a session restriction.  Our look was on the 5000 series and comparing to the 7050 capacity.  The limiting factor for our traffic load was bandwidth.  So we have not enabled the feature.

by MattJamison
on ‎07-23-2014 12:32 PM

Does anyone know if this is supported on a PA-200? If so how many connections?

by ialeksov
on ‎07-23-2014 12:41 PM

SSL decryption is supported on the PA200. There is a limit of 1024 sessions.

I hope this helps you Matt

by MattJamison
on ‎07-23-2014 12:45 PM

It does. The article was written pre-PA-200 so that's why I wasn't sure. Someone might want to add it in there right above the PA-500.

Thanks again!

by oschuler
on ‎07-23-2014 11:43 PM

You can also look it up on the PA homepage under Products - Compare Products. E.g. Product Comparison

by MichalKaprinay
on ‎11-05-2014 05:17 AM

Hi,

might i disable possibility to see capture packets in traffic log, when the traffic is decrypted? problem is that i can see also users passwords in some cases.

>set application dump-unknown no - didn't help

Unknown capture               : off

Thanks

Michal

by bbilut
on ‎01-21-2015 08:11 AM

When I create a self-signed certificate are these the right options to select? I found this in an article outside of Palo Alto. Can somebody shed some light on this part when you create your self-signed cert?

2015-01-21 10_08_14-nadmmdfpa5050-1.png

by Soundshore
on ‎02-23-2015 01:24 PM

Step 3 does not even tell you what to do?! Only tells you what not to do. Poor instructions, please remove.

by andymong
on ‎02-25-2015 04:23 AM

@bbilut. You only select the Forward untrust certificate, nothing else. That way the users are presented with a certificate that they do not trust, just like the ones that PA may connect on the Internet. Unless you plan on distributing that self-signed certificate to the end users's Clients Computers as trusted certificate. Then you would select trusted root ca and Forward trust certificate.

But generally speaking, if you have an internal CA that your users end Computers trust, I would use that as Forward trust instead. and the self-signed as Forward untrust certificate

by
on ‎03-10-2015 11:46 AM

Soundshore,

We are currently modifying this document, and will add/change information, esp the fact that there was no screenshot showing the decryption rules.

by andymong
on ‎03-11-2015 01:02 AM

You may also want to include the fact that ECDHE ciphers aren't supported

by Conor.McAteer
on ‎03-16-2015 02:36 AM

bbilut

regarding the Untrust Certificate

Scenario:  A user opens a webpage without a valid certificate.

If you have SSL decryption enabled, you do not want the PaloAlto firewall to sign the invalid certificate with its your internal CA or self signed cert.  If it did this, the client PC would now see the site as having a valid certificate and would trust the site.

For such sites, you could create a second cert and add in the certificate attributes (for tech savvy users).  Mark this as Forward Untrust certificate.  Do not export this certificate and do not roll it out to any client devices.

Now when a user accesses a site without a valid certificate, the PaloAlto signs it with the Untrust certificate and the client device then still sees it as an Untrusted certificate and will pop up the certificate error.

The PaloAlto also gives you the ability to apply a Decryption Profile to enforce what happens when a user hits a site without a valid certificate.  With a strict decryption profile, you could prevent a user from ignoring the browser's certificate errors and they will not be able to proceed to the site.

by lewis
on ‎03-16-2015 10:06 AM

Is there an easy/simple way to determine how many concurrent ssl connections you have than break it down on url category?

by
on ‎03-18-2015 08:11 AM

Conor, thanks for adding this.

by
on ‎03-18-2015 08:34 AM

Lewis, To see the active sessions that have been decrypted:

> show session all filter ssl-decrypt yes state active

by lewis
on ‎03-19-2015 05:38 AM

Thanks, this is good. But what I am looking for is a easy way to determine the concurrent number of ssl (not decrypted) connections broken down to a per url category.

For example can I see how many ssl connections I have to web-based-email or personal-sites-and-blogs. My goal for this exercise is to understand if our PA is large enough to decrypt all the categories we are looking to decrypt. I know I can pull logs out in to excel to do this but was hoping there was an easier way in the CLI. 

by SDorsey
on ‎03-19-2015 07:03 AM

Not sure I agree on this particular point regarding SSL inspection. In the enterprise, they own the network equipment, endpoints, internet connection etc. So it should be completely up to the company on how they handle the traffic and the risks involved. There is certainly a level in risk in NOT inspecting SSL traffic.

If an end user does not want their personal web browsing decrypted then maybe consider not visiting those personal sites from the corporate network / on the clock.

Now if someone implemented SSL inspection in a public wifi scenario like at a coffee house, I certainly agree privacy would be an issue.

by
on ‎03-19-2015 10:45 AM

Lewis, I do not know the way to do this via the CLI.  The easiest way to gather that data would be through Reports.

by lewis
on ‎03-19-2015 10:49 AM

that is what i am thinking too, just was hoping there was a cli option. thanks

by mobradovic
on ‎04-22-2015 09:46 AM

Good Day

Is there an updated document that show's the current firewall's? This document doesn't have the PA-3000's on it.

Thanks

by
on ‎04-22-2015 02:13 PM

Mobradovic:

I am sorry, I did not get those in there.

I am in the middle of modifying this to have the 3000 series Max number of SSL Decrypt sessions to be added to this list. Thanks for pointing this out.

by MohammadHudaib
on ‎04-22-2015 10:52 PM

Hi,

I implemented SSL decrtyption to VPN tunnels. But it didn't work. Should I apply it to the IKE tunnel (between gateways) or to IPsec tunnel (between end servers) ??

by
on ‎04-23-2015 12:04 PM

I think you need to focus on making sure that the Zones are defined properly in the decryption policy. Most of the time the VPN zone is going to be different than the trust/untrust zones. If that does not help, then please give TAC a call, or open a Support case to have the TAC help.

by EdwinD
on ‎05-04-2015 04:37 PM

The private key matters.   I have a certificate which I have re-issued many, many times.  My certificate authority allows this.  Thus, I have the same certificate loaded on many servers with different private keys.   Though to me these all seem to be my one wild-card certificate, they are truly different certificates.

Thus, to decrypt my servers using ssl-inbound-inspection I must load each and every one of these instances of my certificate onto the Palo Alto Networks Firewall and then configure my Decryption Policy to use the proper certificate for each server.

This took me a little time to figure out.   Hopefully this post helps someone else.  This was the cause of my SSL decryption failures.

With PanOS 5, I do not see a way of supporting SNI.  In other words, I do not see a way to specify different SSL Inbound Inspection certificates for the same destination IP address.   Hopefully we get feature this soon if it isn't already in 6.x.

by jason.crabtree
Friday - last edited Friday

Hello,

I'm on PAN-OS 7.1 and I was going to implement not allowing users to bypass decryption if the hardware limit has been reached, but this command doesn't seem to be valid any more:

"set deviceconfig setting ssl-decrypt deny-setup-failure yes"

Is there and equivalant?

If ther is no equivalent can anyone tell me the default behavior?

 

Thanks,

Register now
Ask Questions Get Answers Join the Live Community