SSL Decryption for Some Site Shows as Not Trusted

Announcements
Attention: Our Customer Support Portal (CSP) is currently experiencing intermittent login disruptions, and we are actively working towards a solution. We appreciate your patience and apologize for any inconvenience this may cause.
Printer Friendly Page

Symptom

A Palo Alto Networks firewall has a list of trusted root Certificate Authorities (CAs), which the firewall uses to check the validity of an SSL site when doing decryption. When CAs change their root certificate, or begin signing server certificates using a new root certificate, the list must be updated.

 

Workaround

Manually install a new root CA, including the newer root CA from GoDaddy and other CA certificates:

  1. For GoDaddy, copy the following GoDaddy root CA (including the BEGIN and END lines) into a plain text editor such as Notepad, (do not use Wordpad, Word, or similar rich-text editors).
    -----BEGIN CERTIFICATE-----
    MIIDxTCCAq2gAwIBAgIBADANBgkqhkiG9w0BAQsFADCBgzELMAkGA1UEBhMCVVMx
    EDAOBgNVBAgTB0FyaXpvbmExEzARBgNVBAcTClNjb3R0c2RhbGUxGjAYBgNVBAoT
    EUdvRGFkZHkuY29tLCBJbmMuMTEwLwYDVQQDEyhHbyBEYWRkeSBSb290IENlcnRp
    ZmljYXRlIEF1dGhvcml0eSAtIEcyMB4XDTA5MDkwMTAwMDAwMFoXDTM3MTIzMTIz
    NTk1OVowgYMxCzAJBgNVBAYTAlVTMRAwDgYDVQQIEwdBcml6b25hMRMwEQYDVQQH
    EwpTY290dHNkYWxlMRowGAYDVQQKExFHb0RhZGR5LmNvbSwgSW5jLjExMC8GA1UE
    AxMoR28gRGFkZHkgUm9vdCBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkgLSBHMjCCASIw
    DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAL9xYgjx+lk09xvJGKP3gElY6SKD
    E6bFIEMBO4Tx5oVJnyfq9oQbTqC023CYxzIBsQU+B07u9PpPL1kwIuerGVZr4oAH
    /PMWdYA5UXvl+TW2dE6pjYIT5LY/qQOD+qK+ihVqf94Lw7YZFAXK6sOoBJQ7Rnwy
    DfMAZiLIjWltNowRGLfTshxgtDj6AozO091GB94KPutdfMh8+7ArU6SSYmlRJQVh
    GkSBjCypQ5Yj36w6gZoOKcUcqeldHraenjAKOc7xiID7S13MMuyFYkMlNAJWJwGR
    tDtwKj9useiciAF9n9T521NtYJ2/LOdYq7hfRvzOxBsDPAnrSTFcaUaz4EcCAwEA
    AaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYE
    FDqahQcQZyi27/a9BUFuIMGU2g/eMA0GCSqGSIb3DQEBCwUAA4IBAQCZ21151fmX
    WWcDYfF+OwYxdS2hII5PZYe096acvNjpL9DbWu7PdIxztDhC2gV7+AJ1uP2lsdeu
    9tfeE8tTEH6KRtGX+rcuKxGrkLAngPnon1rpN5+r5N9ss4UXnT3ZJE95kTXWXwTr
    gIOrmgIttRD02JDHBHNA7XIloKmf7J6raBKZV8aPEjoJpL1E/QYVN8Gb5DKj7Tjo
    2GTzLH4U/ALqn83/B2gX2yKQOC16jdFU8WnjXzPKej17CuPKf1855eJ1usV2GDPO
    LPAvTK33sefOT6jEm0pUBsV/fdUID+Ic/n4XuKxe9tQWskMJDE32p2u0mYRlynqI
    4uJEvlz36hz1
    -----END CERTIFICATE-----
  2. Save the file with a .txt, .cer, or .crt extension.
  3. Go to Device > Certificates and click Import:
    ImportCert.png
  4. Select the file saved from Step 2 and click OK.
  5. Click the name of the new certificate, select Trusted Root CA, and click OK.
    TrustedRoot.png
  6. Commit the changes.

 

Note: Currently, there is no code-level resolution to this issue. Palo Alto Networks is evaluating the best course of action for updating the list of trusted root CAs.

 

owner: gwesson

Comments

Can we get an alert whenever someone tries to visit a site that has an untrusted certificate (either because of expiration, self-signed, or untrusted CA) ?

Would you mind adding the "Go Daddy Root Certificate Authority - G2" per default to the PA Trusted Root Certification Authorities?

Please see Go Daddy Repository, SSL Certificate Information.

Is there any update to this?  I could be mistaken, but isn't the GoDaddy certificate that you reference the same one that the support portal and this very KB Article repository (live.paloaltonetworks.com) uses to sign the HTTPS site?  I'm getting an untrusted-Root-CA error, forcing my Palo Alto Firewalls to use the Forward-Untrust policy for SSL decryption (and generating browser errors) trying to access the Palo Alto site.

Yes, this is the cert that secures live.paloaltonetworks.com.  Follow this article and it solves that issue.  I've ran across several other sites that are secured with this CA, as well. 

It seems that 6.0.5, comes loaded with Go_Daddy_Root_Certificate_Authority_-_G2, and 6.1 does not.

Isn't not having an automatic root certificate update process a major flaw in Palo Alto Networks implementation of forward SSL decryption?

If I follow the steps outlined here, then the certificate authority above becomes compromised and removed from the trusted list of root authorities by Microsoft, Apple, Mozilla, Chrome etc I would never know because my Palo Alto accepts the root CA and my browsers have no idea I'm talking to a compromised server through some MITM attack since the traffic is all signed by my Palo Alto certificate.

For a real world example, Google the term ​Google boots China's main digital certificate authority CNNIC | ZDNet.

  • Google is blacklisting all digital certificates from the China Internet Network Information Center (CNNIC), the organisation that manages the .cn domain and a widely trusted root certificate authority.
  • Google, Mozilla, and Microsoft last week responded to a mis-issued digital certificate from an Egyptian company called MCS Holdings, which could have allowed an attacker to impersonate a Google site and intercept traffic to and from it.

This is just one of many examples. 

For each and every certificate I mark as a trusted root in my Palo Alto Networks Certificates tab, I become less secure.  Cross signed certificates can be revoked by any of the parent signers.  You see this a lot with EV certs with multiple chains of trust.   If I trust an intermediate because the Palo Alto firewall doesn't have the CA installed and subequently fails on the CRL/OCSP check and blocks SSL traffic then an enormous amount of this CRL/OCSP is made useless.

I beg Palo Alto to remediate this security issue by adding an auto update process.  I have KB2677070 installed thus Win7 updating these lists, Chrome updating, Firefox updating and so on.   But none of this  does me any good because all my HTTPS traffic is signed by the Palo Alto certificate authority.

If I'm wrong, please say so.   My testing shows I'm not.

For testing, you should NOT be able to get to these websites.  If you can, your installation is insecure.

Verisign test site: Untitled Document

GRC test site GRC's | Security Certificate Revocation Awareness Test 

Because I have revoked the CNNIC CA on my PC, I should not be able to visit: https://www1.cnnic.cn/

With Palo Alto Network decryption disabled, I am receive the proper warning that CNNIC.CN is not trusted.   With Palo Alto Networks decryption disabled, I do not.   This is just one example.

This is why it is so very important:

http://news.netcraft.com/archives/2014/04/24/certificate-revocation-why-browsers-remain-affected-by-...

https://www.grc.com/revocation/crlsets.htm

Any update on the automatic update of root certificates?

I would like to see this implemented.

Using the SSL Forward Untrust Certificate to sign trusted Certificates is one of my main issues when implementing the Decryption feature.

Although some sites are signed with a Root CA Certificate that are inIcuded in the "Default Trusted Certificates Authorities" of PA, still PA uses my SSL Forward Untrust Certificate to sign.

I have tried to manually import the Root CA Certificate and all the relevant CA Certificates in the SSL chain of trust to the "Device Certificates" and still facing the same problem.

 

One specific example is: https://www.paloaltonetworks.com which is originally signed by CA "Verizon Akamai SureServer CA G14-SHA2" wich is signed by the "Root CA Baltimore CyberTrust Root". When applying Decryption PA signed the Certificate with my SSL Forward Untrust Certificate although "Root CA Baltimore CyberTrust Root" is included in the "Default Trusted Certificates Authorities" of PA.

 

Anyone else experiencing the same issue?

 

Any updates to this thread with a solution?

 

I wholeheartedly agree with EdwinD, at minimum this is an annoyance, at maximum, this is a major security flaw.

 

Having the same issue. Need to add missing root CA once in a while which is very weird to do on a leading security product.

Can this be updated via content updates?

Hi @YuvalBenAri, have you raised this with support? Which root CAs are you adding, mainstream or smaller parties? Which PAN-OS are your systems on?

I will contact support, but I am guessing the answer will be to raise a feature request :)

I am on PanOS 7.1, but it is also missing on Panorama with 8.0

So far I had to add a GeoTrust, GlobalSign and GoDaddy certificates. It seems that mostly intermediate certs. are missing.

 

 

How do you get other providers root CAs to import them into the PA? I am having an issue with Network Solutions OV Server CA 2 certs. I see Network Solutions CA in the default trusted list but assume there is something different about this one. 

 

Also DigiCert SHA2 Extended Validation Server CA.

The CA certs are public, you can just get them from their website or just browse to the site using a browser and download it using the browser. You can download the cert from the PanOS and open it locally to compare to the above cert and see if they are different or just try to import the OV cert and see if it helps. The problem might also be a missing intermiddiate certificate which can be a bit more tricky to find.

Absolutely ridiculous.  This is a YUGE oversight by development.  I'm importing at least a cert chain a day.  It necessary to import the entire chain in most cases since the firewall it's smart enough to chain intermediate certs to root certs already installed in its default cert store.  

Hi @ice-quake

 

If you need to import this many cert chains a day I suggest you open a support case to have this behavior investigated. The article above should assist with occasional issues, not for your volume of operations. There may be a bigger issue

@reaper

 

I did open a case with support.  The answer is "there is a feature request pending."  It's not the result of a bigger issue.  The firewall simply doesn't trust the cert b/c the CA cert isn't instaled in the default trusted cert authority store on the device.  The ultimate irony - yesterday I had to import some new GoDaddy certs when Palo Alto updated/renewed the certs for its support portal (supportcases.paloaltonetworks.com).   I was getting the "your connection is not private message" in my browser b/c the firewall signed the support portal certs with the forward untrust cert.  This is a development issue.  Plain and simple.  The certs should be updated via content updates not OS updates.

 I also requested this feature from support. No idea when it might happen...

Feature requests are actually handled by sales teams, support (or the community team) cannot submit feature requests so please reach out to your sales team if you want to add your vote to a feature request

 

Are these issues caused because the sites being accessed don't have intermediate trust certificates instralled on them? Edit - nope, the GRC site not being served Foward Untrust Certificates.

Not sure if anyone is still looking at the comments on this, I know it is a bit of a necro thread. Turns out you can turn on revocation status verification, it is off by default. There is some overhead involved. https://www.paloaltonetworks.com/documentation/80/pan-os/pan-os/certificate-management/set-up-verifi...