pandb-database will not install on Pan_OS 9.0.x

[BruceBennett]

I have a couple of firewalls that are running Pan-OS 9.0.3 that I cannot get the pandb-database to install and update. At least I cannot prove that it is downloading and active. Until 9/30/19, the 9.0 docs for this were the same as the 8.1 docs. According to the new docs, it looks like PANDB is active, but when I check the status on 9.0, it shows a DB version of 0000.00.00.000. Below are the results from the command "show url-cloud status" for my 9.0 and 8.1 firewalls.

9.0

PAN-DB URL Filtering
License : valid
Cloud connection : not connected
URL database version - device : 0000.00.00.000
URL protocol version - device : pan/0.0.2

8.1

PAN-DB URL Filtering
License : valid
Cloud connection : not connected
URL database version - device : 20190909.20113
URL protocol version - device : pan/0.0.2

When I look in monitor for the 8.1 firewall, I can see url-categories. On the 9.0 firewall, the only url-categories I get are "any" and "not-resolved". That is another indicator that is showing that I do not have a url-database.

I have tried several things, especially when the 9.0 docs were not up to date, including trying to manually install using "request url-filtering install pandb-database", which fails because there is no image uploaded. But, if you try to download the database using the "request url-filtering download" command, you can no longer specify "request url-filtering download paloaltonetworks region North-America", in 9.0 once you get to download in the command, the only options available (just pressing tab) comes out with "request url-filtering download status vendor brightcloud". 

I have the exact same issue on two different 9.0 firewalls, so it does not seem to be something with the firewall, and I have tried this with 9.0.3 and currently with 9.0.3-h3, same results.

Unfortunately, I cannot take the firewalls down to 8.1.x and then run the update/download because we are using GRE tunnels on these firewalls, so that is not a viable option.

[BruceBennett · Accepted solution]

An update for anyone that looks this up later. This is now solved.

We have a mixed environment that is very restrictive, including SSL intercept. In the system logs, we were seeing "CURL ERROR: SSL certificate problem: self signed certificate in certificate chain". In the CLI we were seeing the following with the "tail follow yes mp-log devsrv.log" command:

2019-10-29 22:42:03.319 +0000 Warning: pan_cloud_agent_get_curl_connection(pan_cloud_agent_connect.c:2609): cannot elect a cloud
2019-10-29 22:42:03.319 +0000 curl error: SSL certificate problem: self signed certificate in certificate chain
2019-10-29 22:42:03.320 +0000 Failed to open connection with the cloud after 12360 consecutive tries.
2019-10-29 22:42:04.420 +0000 CLOUD_ELECTION: in wait_t 0 secs.
2019-10-29 22:42:04.544 +0000 Error: verify_cb(pan_ssl_curl_utils.c:614): Error with certificate at depth: 2
2019-10-29 22:42:04.544 +0000 Error: verify_cb(pan_ssl_curl_utils.c:616): Basic Validation of x509 cert Fail ; Code : 19

...

2019-10-29 22:42:04.544 +0000 Error: verify_cb(pan_ssl_curl_utils.c:625): Failed to validate x509 cert from ctx: (19) self signed certificate in certificate chain
2019-10-29 22:42:04.544 +0000 Warning: pan_cloud_agent_collect_cloud_info_cb(pan_cloud_agent_connect.c:1957): cloud elect connection close

I found the available docs were not complete for the changes between 8.x and 9.x PanDB process.

In the end, I found that 8.x and 9.x are using different destinations for updates and since 9.x does not use a seed file, it has to reach the destination to get everything going.

Here are the destinations that are being used for both:

8.x PanDB - 65.154.226.123 "dl1prod.urlcloud.paloaltonetworks.com"
9.x PanDB - 65.154.226.124 "pandb2qa.urlcloud.paloaltonetworks.com" See message from Ldemos below for the proper URL.

In the end, it was the SSL intercept that was keeping the access to pandb2qa.urlcloud.paloaltonetworks.com from working. Once SSL intercept was removed everything started working within the next 10 minutes.

While you can turn off the "verify update server identity" for updates, this does not appear to be an option with the PanDB access, the certificate chain is verified and will not work with an SSL intercept in the middle.

Hopefully I put in enough key words for this to be found if someone else is running into this issue.

[ldemos]

Please avoid using (pandb2qa.urlcloud.paloaltonetworks.com) for PAN-DB URL Filtering cloud server setting.  This server was intended for internal testing.  
Customers should NOT be utilizing this server because it's for internal testing, there will be data consistency problems and uptime problems.  The server will also be disabled soon.  Please use the universal server [serverlist.urlcloud.paloaltonetworks.com].

Thank You

[BruceBennett]

I was able to update the post and it now refers to your post.