Not-resolved URL blocking PAN url cloud updates

[drewdown]

I am in a pickle, I have PANs managed by panorama but I can't push any URL updates to the PAN that is blocking itself.  Can I just update that policy that this traffic is hitting and remove the URL category action on it?  Will that allow it to connect? I tried updating service routes to use the outside interface but still URL updates are not happening and it looks to be because the new license was installed on the 18th which in turn broke this someway.

You can see below the screenshot from the log timestamps line up with the logs from CLI but after 13:48 its still broken but not being logged.  I think that is after I changed DNS/NTP/PA Network Services and URL updates to use the outside interface.  But still no joy in getting this working.

(active)> show log system direction equal backward receive_time in last-24-hrs | match PAN-DB
2023/07/19 15:48:59 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:Couldn't connect to server).
2023/07/19 15:19:25 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:Couldn't connect to server).
2023/07/19 14:49:52 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:Couldn't connect to server).
2023/07/19 14:20:18 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:Couldn't connect to server).
2023/07/19 13:50:44 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:Couldn't connect to server).
2023/07/19 13:45:43 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:Couldn't connect to server).
2023/07/19 13:43:28 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 13:43:27 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:Couldn't resolve host name).
2023/07/19 13:29:35 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 13:09:23 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 12:49:10 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 12:28:58 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 12:14:39 info     url-fil        url-bac 0  Backup of PAN-DB finished successfully.
2023/07/19 12:08:45 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 11:48:32 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 11:28:19 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 11:08:07 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 10:47:53 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 10:27:41 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 10:07:29 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 09:47:15 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 09:27:03 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 09:06:51 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 08:46:39 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 08:26:26 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 08:14:38 info     url-fil        url-bac 0  Backup of PAN-DB finished successfully.
2023/07/19 08:06:13 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 07:45:59 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 07:25:45 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 07:05:31 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 06:45:18 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 06:25:06 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 06:04:55 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 05:44:42 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 05:24:28 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 05:04:15 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 04:44:03 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 04:23:50 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 04:14:37 info     url-fil        url-bac 0  Backup of PAN-DB finished successfully.
2023/07/19 04:03:38 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 03:43:25 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 03:23:13 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 03:03:01 medium   url-fil        url-dow 0  PAN-DB cloud list loading failed (ERROR:SSL connect error).
2023/07/19 00:14:35 info     url-fil        url-bac 0  Backup of PAN-DB finished successfully.
2023/07/18 20:14:34 info     url-fil        url-bac 0  Backup of PAN-DB finished successfully.
2023/07/18 16:14:33 info     url-fil        url-bac 0  Backup of PAN-DB finished successfully.


(active)> delete license key
  Advanced_URL_Filtering_2023_07_18_94880943.key   2023/07/19 07:40:31        0.3K

(active)> show url-cloud status

PAN-DB URL Filtering
License :                          valid
Cloud connection :                 not connected
URL database version - device :    0000.00.00.000
URL protocol version - device :    pan/0.0.

(active)> ping host s0000.urlcloud.paloaltonetworks.com
PING s000new.urlcloud.paloaltonetworks.com (35.244.200.72) 56(84) bytes of data.
64 bytes from 72.200.244.35.bc.googleusercontent.com (35.244.200.72): icmp_seq=1 ttl=55 time=17.9 ms
64 bytes from 72.200.244.35.bc.googleusercontent.com (35.244.200.72): icmp_seq=2 ttl=55 time=17.9 ms

[drewdown]

Welp..

1. I changed all those service routes to use my MGMT interface (didn't work using outside interface or any other one)
2. I added pan-db-cloud to my  list of allowed APPs
3. I changed the unresolved category to ALERT vs block/block. 
4. I deleted all the old license key files from the CLI

Why after 6+ years I had to do all this I have no idea..I can't say what broke or what fixed it but its working again.  

PAN-DB URL Filtering
License :                          valid
Current cloud server :             serverlist3.urlcloud.paloaltonetworks.com
Cloud connection :                 connected
Cloud mode :                       public
URL database version - device :    20230719.20330
URL database version - cloud :     20230719.20330  ( last update time 2023/07/19 16:38:48 )
URL database status :              good
URL protocol version - device :    pan/2.0.0
URL protocol version - cloud :     pan/2.0.0
Protocol compatibility status :    compatible

[Adrian_Jensen · Accepted solution]

Yes, I have had this happen before. The default URL Filtering profile action for "unknown" and "not-resolved" is allow, but I suspect many people setup custom URL Filtering profiles to block or continue for additional security. When you upgrade from the PAN-DB database to URL-Cloud database (8.x to 9.x) the database is defaulted and must be repopulated from the cloud. I have also had the URL-Cloud database mysteriously reset and need to re-initialize. Unfortunately when this happens the URLs needed to initialize the database become "not-resolved" and are blocked in your custom URL Filter...

To handle this startup case I added "*.urlcloud.paloaltonetworks.com/" to a Custom URL Category object that always allows in my custom URL Filtering profiles. Since Custom URL Categories are defined outside of URL-Cloud they always resolve, and that allows the *.urlcloud.paloaltonetworks.com update addresses to pass URL Filtering, even when the URL-Cloud database is uninitialized or broken.

[drewdown]

Which is what we did, un-resolved was set to block and I believe PA told me to do that but when doing that you can be left in the lurch like I was.  No upgrade was done of late as all of my PANs are running 9.1.14-h as that is the latest version the majority of them support.  I am going to take your suggestion and add *.urlcloud.paloaltonetworks.com/ to my profiles.

Another odd thing I noticed, all of my URL category/Filters are from Panorama but when I make a change to them in the BRANCHES level I don't get the option to push it to my firewalls.  Only COMMIT.   I see whatever change I made in those device groups but again no way to PUSH it.  So right now something is broken that won't let me push any URL category changes down to the firewalls from Panroama.  I also see Panorama shows a different URL category then what the local FWs show via CLI.  So something is wrong here and I can't seem to figure out what that is.  

Anyone know why that is?