How to Implement and Test SSL Decryption

Printer Friendly Page

Overview

PAN-OS can decrypt and inspect inbound and outbound SSL connections going through the Palo Alto Networks firewall. SSL decryption can occur on interfaces in virtual wire, Layer 2 or Layer 3 mode by using the SSL rulebase to configure which traffic to decrypt. In particular, decryption can be based upon URL categories and source user and source/target addresses. Once traffic is decrypted, tunneled applications can be detected and controlled, and the decrypted data can be inspected for threats, URL filtering, file blocking, or data filtering. Decrypted traffic is never sent off the device.

 

Inbound SSL Decryption

In the case of inbound traffic to an internal Web Server or device, the administrator imports a copy of the protected server’s certificate and key. When the SSL server certificate is loaded on the firewall, and a SSL decryption policy is configured for the inbound traffic, the device then decrypts and reads the traffic as it forwards it. No changes are made to the packet data, and the secure channel is from the client system to the internal server. The firewall can then detect malicious content and control applications running over this secure channel.

 

Outbound SSL Decryption (SSL Forward Proxy)

In this case, the firewall proxies outbound SSL connections by intercepting outbound SSL requests and generating a certificate on the fly for the site the user wants to visit. The validity date on the PA-generated certificate is taken from the validity date on the real server certificate.

The issuing authority of the PA-generated certificate is the Palo Alto Networks device. If the firewall’s certificate is not part of an existing hierarchy, or is not added to a client’s browser cache, then the client receives a warning when browsing to a secure site. If the real server certificate has been issued by an authority not trusted by the Palo Alto Networks firewall, then the decryption certificate is using a second “untrusted” Certificate Authority (CA) key to ensure the user is warned of any subsequent man-in-the-middle attacks.

 

To configure SSL decryption:

  1. Configure the firewall to handle traffic and place it in the network.
  2. Make sure the proper Certificate Authority (CA) is on the firewall.
  3. Configure SSL decryption rules.
  4. Enable SSL decryption notification page (optional).
  5. Commit changes and test decryption.

 

Steps

1. Configure the firewall to handle traffic and place it in the network

Make sure the Palo Alto Networks firewall is already configured with working Interfaces (Virtual Wire, Layer 2 or Layer 3), Zones, Security Policy and already passing traffic.

 

2. Load or Generate a CA certificate on the Palo Alto Networks firewall

A Certificate Authority (CA) is required to decrypt traffic properly by generating SSL certificates on the fly. Create a self-signed CA on the firewall or import a Subordinate CA (from your own PKI infrastructure). Select "Forward Trust Certificate" and "Forward Untrust Certificate" on one or more certificates to enable the firewall to decrypt traffic.

Note: Because SSL Certificate providers like Entrust, Verisign, Digicert, and GoDaddy do not sell CAs, they are not supported in SSL Decryption.

 

From the firewall GUI, go to Device > Certificates. Load or generate a certificate for either inbound inspection or outbound (forward proxy) inspection.

 

Generating a Self-Signed Certificate

Using a Self-Signed Certificate is recommended. For information on generating a Self-Signed Certificate, please see:

How to Generate a New Self-Signed SSL Certificate

 

Generating and Importing a Certificate from Microsoft Certificate Server

  1. On the Microsoft Certificate Server for your organization, request an advanced certificate using certificate template “subordinate CA”. Download the cert.
  2. After downloading, export the certificate from the local certificate store. In IE, access the Internet Options dialog, select the Content tab, then click the Certificates button. The new certificate can be exported from the Personal certificates store. Select “Certificate Export Wizard”, export the private key, then select the format. Enter a passphrase and a file name and location for the resulting file. The certificate will be in a PFX format (PKCS #12).
  3. To extract the certificate, use this openSSL[4] command:
    openssl pkcs12 –in pfxfilename.pfx –out cert.pem –nokeys
  4. To extract the key, use this openSSL command:
    openssl pkcs12 –in pfxfilename.pfx –out keyfile.pem -nocerts
  5. Import the cert.pem file and keyfile.pem file into the Palo Alto Networks firewall on the Device tab > Certificates screen.
  6. In the case of a High Availability (HA) Pair, also load these files into the second Palo Alto Networks firewall, or copy the certificate and key via the High Availability widget on the dashboard.

 

The "Forward Trust" and "Forward Untrust" certificates:

cert-info-ssl-decrypt.png

 

 

Note: If using a self-signed CA, export the public CA Certificate from the firewall and install the certificate as a Trusted Root CA on each machine's browser to avoid Untrusted Certificate error messages inside your browser. Network administrators usually use GPO to push out this certificate to each workstation.

 

Examples of browser errors if the self-signed CA Certificate is not trusted:

 

Firefox untrusted CA error:

ssl-decrypt1.png

 

Chrome untrusted CA error:

ssl-decrypt2.png

 

Internet Explorer untrusted CA error:

ssl-decrypt3.png

 

3. Configure SSL Decryption Rules

The network administrator determines what needs to be decrypted. A few suggestions for configuring SSL decryption rules:

  • Implement rules in a phased approach. Start with specific rules for decryption, and monitor the typical number of SSL connections being decrypted by the device.
  • Avoid decrypting the following URL categories, as users may consider this an invasion of privacy:
    • Financial services
    • Health and medicine
  • Do not decrypt applications where the server requires client-side certificates (for identification).
    • You can either block or allow connections requiring client authentication via the decryption profile feature introduced in PAN-OS 5.0.

 

An example of an outbound rulebase following suggestions for decryption.

2015-03-10 12_45_40-77B-PA-VM-100.png

4. Enable SSL Decryption Notification web page (optional)

  • The user can be notified that their SSL connection will be decrypted using the response page found on the Device tab > Response Pages screen. Click "Disabled," check the "Enable SSL Opt-out Page" option and click OK.

Untitled.jpg

 

The default SSL Opt-out page page can be exported, edited via an html editor, and imported to provide company-specific information:

1.png

5. Test Outbound Decryption

To test outbound decryption:

  • Make sure that in the outbound policy, the action is to alert for any viruses found. Also enable packet capture on that anti-virus security profile. Commit any changes made.
  • On a PC internal to the firewall, go to www.eicar.org. In the top right corner:

Untitled.jpg

  • Click “Download anti-malware testfile."
  • In the screen that appears, scroll to the bottom.
  • Download the eicar test virus using http. Any of the these four files will be detected.

Untitled.jpg

  • Go to the Monitor tab > Threat log, and look for the log message that detects the eicar file.

ssl-eicar-log1.png

  • Click the green arrow in the column on the left to view the captured packets.
    ssl-eicar-log2.png

ssl-eicar-cap1.png

  • Click the magnifying class in the far left column to see the log detail.

ssl-eicar-log3.png

  • Scroll to the bottom, and look for the field “Decrypted.” The session was not decrypted:

ssl-eicar-log-detail-nodecryp1.png

 

  • Go back to the www.eicar.org downloads page. This time use SSL enabled protocol HTTPS to download the test virus.

Untitled.jpg

  • Examine the Threat logs. The virus should have been detected, since the SSL connection was decrypted. A log message that shows Eicar was detected in web browsing on port 443 will be visible.
    ssl-eicar-log4.png

  • View the packet capture (optional) by clicking the green arrow.

ssl-eicar-cap2.png

  • To the left of that log entry, click the magnifying class. Scroll to the bottom. Under Flags, check to see that “Decrypted” is checked:

ssl-eicar-log-detail-decryp1.png

 

The virus was successfully detected in an SSL-encrypted session.

 

To test the “no-decrypt” rule, first determine what URLs fall into financial services, shopping, or health and medicine categories. For BrightCloud, go to http://www.brightcloud.com/testasite.aspx. For PAN-DB, use Palo Alto Networks URL Filtering - Test A Site , and enter a URL to see what the category is.

Once web sites that are classified into categories that will not be decrypted are found, use a browser to go to those sites using https. There should be no certificate error when going to those sites. The web pages will be displayed properly. Traffic logs will show the sessions where application SSL traverses port 443, as expected.

 

To Test Inbound Decryption:

  • Examine the traffic logs dated before enabling SSL for inbound decryption on the firewall. Look at traffic targeted for the internal servers. In those logs, the application detected should be “ssl" going over port 443.
  • From a machine outside of the network, connect via SSL to a server in the DMZ. There will be no certificate errors, as the connection is not being proxied, just inspected.
  • Examine the logs for this inbound connection. The applications will not be “ssl" but the actual applications found inside the SSL tunnel. Click the magnifying glass icon in those log entries to confirm decrypted connections.

decrypted.png

 

Helpful CLI Commands:

To see how many existing SSL decryption sessions are going through the device:

> debug dataplane pool statistics | match proxy

 

Output from a PA-2050, where the first command shows 1024 available sessions, and the output of the second command shows five SSL sessions being decrypted (1024–1019=5):

admin@test> debug dataplane pool statistics | match proxy

[18] proxy session            :    1019/1024    0x7f00723f1ee0

 

To see the active sessions that have been decrypted:

> show session all filter ssl-decrypt yes state active

Maximum number of concurrent SSL decrypted sessions in PAN-OS 4.1, 5.0, 6.0 and 6.1 (both directions combined):

Hardware SSL Decypted Session Limit
VM-100 1,024 sessions

VM-200

1,024 sessions
VM-300 1,024 sessions
PA-200 1,024 sessions
PA-500 1,024 sessions
PA-2020 1,024 sessions
PA-2050 1,024 sessions
PA-3020 7,936 sessions

PA-3050

15,360 sessions
PA-3060 15,360 sessions
PA-4020 7,936 sessions
PA-4050 23,808 sessions
PA-4060 23,808 sessions
PA-5020 15,872 sessions
PA-5050 47,616 sessions
PA-5060 90,112 sessions
PA-7000-20G-NPC 131,072 sessions
PA-7050 786,432 sessions

 

If the limit is reached, all new SSL sessions go through as undecrypted SSL. To drop any new SSL sessions beyond the session limit of the device:

> set deviceconfig setting ssl-decrypt deny-setup-failure yes

To check if there are any sessions hitting the limit of the device:

> show counter global name proxy_flow_alloc_failure

To view the SSL decryption certificate:

> show system setting ssl-decrypt certificate
Certificates for Global

SSL Decryption CERT
global trusted
ssl-decryption x509 certificate
version 2
cert algorithm 4
valid 150310210236Z -- 210522210236Z
cert pki 1
subject: 172.16.77.1
issuer: 172.16.77.1
serial number(9)
00 b6 96 7e c9 99 1f a8  f7                      ...~.... .
rsa key size 2048 siglen 2048
basic constraints extension CA 1

global untrusted
ssl-decryption x509 certificate
version 2
cert algorithm 4
valid 150310210236Z -- 210522210236Z
cert pki 1
subject: 172.16.77.1
issuer: 172.16.77.1
serial number(9)
00 b6 96 7e c9 99 1f a8  f7                      ...~.... .
rsa key size 2048 siglen 2048
basic constraints extension CA 1

 

To view SSL decryption settings:

> show system setting ssl-decrypt setting

vsys                          : vsys1
Forward Proxy Ready          : yes
Inbound Proxy Ready          : no
Disable ssl                  : no
Disable ssl-decrypt          : no
Notify user                  : no
Proxy for URL                : no
Wait for URL                  : no
Block revoked Cert            : yes
Block timeout Cert            : no
Block unknown Cert            : no
Cert Status Query Timeout    : 5
URL Category Query Timeout    : 5
Fwd proxy server cert's key size: 0
Use Cert Cache                : yes
Verify CRL                    : no
Verify OCSP                  : no
CRL Status receive Timeout    : 5
OCSP Status receive Timeout  : 5
Block unknown Cert            : no

For a list of resources about SSL Decryption, please refer to the following:

SSL Decryption Quick Reference - Resources

 

For more information on supported Cipher Suites for SSL Decryption, please refer to the following:

SSL Decryption Not Working Due to Unsupported Cipher Suites

Limitations and Recommendations While Implementing SSL Decryption

How to Identify Root Cause for SSL Decryption Failure Issues

 

Note: If anything else needs to be added to this document, please comment below.

 

owner: jdelio

Comments

@reaper ues all certificates are imported correctly as you can see from image that i have a green lock icon in the address bar...TAC spent 1-2 hrs on it but to no avail....not even ciphers issue...

it would be great if you could share more information on the packetcapture

what is the client proposing, what is the server counterproposing and what are the error messages

@reaper

 

here is the full packet capture....select TLS as filter selected and see Encrypted Alerts

 

Packet Capture

@ Joe Delio (and/or others who tried),

 

is it possible to install/use certificate from a known trusted CA (GoDaddy, Thawte, etc) for SSL Forward Proxy?

The point/idea is to make the PA appliance's cert used for "Forward Trust", "Forward Untrust" naturally trusted by clients on internal network/zone (without importing that cert/hierarchy certs into its trusted root CA list).

 

Thanks in advance!

@ AlexanderA

 

Please consider the following documentation, if this is what you are looking for then Public CA cannot be used for SSL Forward Proxy decryption.

 

https://live.paloaltonetworks.com/t5/Management-Articles/SSL-certificates-resource-list/ta-p/53068

 

 

@AlexanderA

I hear this question all the time.. "Can I use a purchased SSL certificate (Entrust, Verisign, etc.) to use for SSL Forward Proxy?" 

 

The answer is:

"Because SSL Certificate providers like Entrust, Verisign, Digicert, and GoDaddy do not sell CAs, they are not supported in SSL Decryption." (which is a note in Step 2 above).

 

Additional info:

A CA (Certificate Authority) is required to make SSL Forward Proxy work (Inbound decryption) because SSL certificates have to be generated in real time. The only way to do this is with a CA. And again, there is no way that Entrust or any other 3rd Party SSL provider will sell you a CA that allows you to create SSL certificates on their behalf.  

Please ask if you are confused.

Has anyone found the location of detailed SSL inspection logs? I've had a poke around the GUI and mp-logs but not found what I'm after. For example, a website which is using what appears to be a valid certificate is getting issued with the forward untrust certificate by the firewall and I want to know why. The correct signing CA (Comodo) is on the firewall yet the decision is made to use the untrust rather than the trust cert. I guess this kind of info must be recorded somewhere...

Hey all - anyone have any issues with getting a Mac to deal with SSL decryption?  I'm able to get the CA inserted into the keyfile and Chrome and Firefox work fine, but when I run the updater under the App Store, I get the error that implies that the App Store is trying to use a certiifate that isn't what it claims to be meaning that the root CA that I inserted into the keyfile isn't being read.  Anyone with any ideas around this?  If someone can point me in any kind of direction here, I'd happily do the testing to make sure it works and we can all get the documentation on this.  Thanks!! 

@cwbuege

 

App Store cannot be decrypted due to pinnned-cert therefore it has to be excluded from the decryption. 

If your Palo is running on PAN-OS 8.0.x the App Store is excluded by default, if the version is older then you will need to add the following links the exclusion:

 

*.itunes.apple.com

itunes.apple.com 

 

Marek 

@MarekWalczak

 

Thank you very much for the quick reply and the information.  Exactly what I've been looking for.  I am running PAN-OS 8.0.6 specifically, but I don't see where the exclusion would be by default.  I have no problem adding the links for exclusions and will test that tomorrow.

 

Other than your giving me this piece of information, is there anywhere else I can do to get this type of information 'officially'?  Basically I'm wondering what other scenarious that Palo's already run into like this that I would want to add to my system to help my users from running into problems like this.  Thanks again!!

 

- Charles

@ausafali88

 

I was looking through these post and I'm experiencing the exact same issue that you described... What did you do to get this solved?

 

- David

 

Nice and detail article .

@dhammel, We recommend opening a case with support if you do not already have one opened, as support will be able to work with the engineering department if needed to help resolve any advanced issues.

Why PA didn't show block page for SSL traffic after drop/reset ?

I was tried but it showed as bellow:aa.JPG

 

Thank you

dat

hi @TranTienDat do you have SSL decryption configured?

Hi @reaper

 

yes, i configured decryption already.

 

when I disable ssl decryption, anydesk client can work correctly.

 

Thanks

dat

Hi!

 

This is good and simple example for getting your Enterprise CA generated certificate for SSL decryption: https://www.paloaltonetworks.com/documentation/80/pan-os/pan-os/decryption/configure-ssl-forward-pro...

Though it misses one important thing: certificate template that must be used for CSR signing is Subordinate Certificate Authority (not Web Server or something similar) in Microsoft CA. But it is at least mentioned in current article and thus provides good information to those end up in KB articles while setting up SSL decryption (I am talking about SSL Forward Proxy).

 

Reaper and jdelio, maybe you can ask PAN documentation creators to add this information (if you verify yourself also that this the case).

Guys , 

 

First of all thanks a lot for wonderful article and very good further discussions on it.

I have a query if we can use third party CA certificate for SSL decryption in panos 8.0.10 ? or I still need to use self signed or internal PKI ?

Hi, kchopra01!

 

What do you mean with third party CA certificate?

If you are referring to public certificate providers then it is not possible as also mentioned in this article: “Note: Because SSL Certificate providers like Entrust, Verisign, Digicert, and GoDaddy do not sell CAs, they are not supported in SSL Decryption.”

Hi @mart_e , Yes I actually was going through panos guide in which under topic 'decryption' , it is written that it is always recommended to use enterprise certificate instead of self signed , so I think I am mixing this word "enterprise" with "public CA".

 

Does enterprise certificate here means  internal certificate of company ? not public CA right ?

please advise.


Hi, @kchopra01!

 

If you are referring to this link I provided few comments back and for this part "(Recommended Best Practice) Enterprise CA-signed Certificates" then yes, it is meant your company CA.

 

So you can use your enterprise/company (Subordinate) CA for SSL decryption as you can generate this yourself- and as mentioned you can not use public CA (like Entrust, Verisign, Digicert, GoDaddy etc) provided CA certificates as they do not sell CAs.