URL filtering database updates problem

[adminglu]

I have a feeling that the URL filtering database updating doesn't work correctly. After a reboot of the firewall it update a few days and then stops. In the URL filtering logs I also see a lot of "not-resolved" even for url's like play.google.com. I've had this with at least PAN-OS 10.2.4-h2 and now also with 11.0.2-h2. When I run the command "show url-cloud status" it seems to be working, accept the last update time:

PAN-DB URL Filtering
License :                          valid
libcurl resolver :                 threaded
Current cloud server :             serverlist2.urlcloud.paloaltonetworks.com
Cloud connection :                 connected
Cloud mode :                       public
URL database version - device :    20231111.20140
URL database version - cloud :     20231111.20140  ( last update time 2023/11/11 10:16:12 )
URL database status :              good
URL protocol version - device :    pan/2.0.0
URL protocol version - cloud :     pan/2.0.0
Protocol compatibility status :    compatible

The management interface can communicate with the Palo Alto servers and I see no blocked traffic. Has anybody seen similar issues and is there a service I can restart to get it working again? Rebooting the firewall isn't realy an option.

Thanks in advance.

[aleksandar.astardzhiev]

Hi @adminglu ,

10.2 and 11.0 are fairly new versions any you maybe hitting some known or unknown bug.

It will be better to submit a ticket to TAC to have a look.

[Adrian_Jensen]

I have seen 2 issues that match the description.

1. I have seen the URL database seeming reset and have to re-initialize, which may take several minutes.
2. There is a known issue specifically with Google search URLs (including play.google.com/) and URL filtering. Apparently a change was made that overwhelmed the URL cloud database. PaloAlto put in a workaround but it seems that sometimes that workaround misses or they are pushing updates which temporarily cause not-resolved again. Have seen the issue intermittently for months.

[adminglu]

Hi @aleksandar.astardzhiev ,

We have a ticket with our support parter, but so far there is no solution of a root cause found. If there is no solution soon I will have them excalate it to TAC.

[adminglu]

We have now deleted the license, then rebooted the firewall and then re-applied the license. We will now continue to watch the URL-DB version to see if it keeps updating. When it is broken we only see URL DB Backup events, and after a reboot we also see URL DB update events every few minutes. This is the log when broken:

show log system direction equal backward receive_time in last-7-days | match PAN-DB
2023/12/15 12:58:48 info url-fil url-bac 0 Backup of PAN-DB finished successfully.
2023/12/15 08:58:42 info url-fil url-bac 0 Backup of PAN-DB finished successfully.
2023/12/15 04:58:36 info url-fil url-bac 0 Backup of PAN-DB finished successfully.
2023/12/15 00:58:31 info url-fil url-bac 0 Backup of PAN-DB finished successfully.
2023/12/14 20:58:25 info url-fil url-bac 0 Backup of PAN-DB finished successfully.
2023/12/14 16:58:19 info url-fil url-bac 0 Backup of PAN-DB finished successfully.
2023/12/14 12:58:13 info url-fil url-bac 0 Backup of PAN-DB finished successfully.

And this is after a reboot:

show log system direction equal backward receive_time in last-7-days | match PAN-DB
2024/01/09 15:28:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20218.
2024/01/09 15:24:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20217.
2024/01/09 15:20:01 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20216.
2024/01/09 15:16:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20215.
2024/01/09 15:12:02 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20214.
2024/01/09 15:08:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20213.
2024/01/09 15:04:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20212.
2024/01/09 15:00:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20211.
2024/01/09 14:56:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20210.
2024/01/09 14:52:01 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20209.
2024/01/09 14:48:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20208.
2024/01/09 14:44:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20207.
2024/01/09 14:40:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20206.
2024/01/09 14:36:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20205.
2024/01/09 14:32:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20204.
2024/01/09 14:27:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20203.
2024/01/09 14:24:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20202.
2024/01/09 14:19:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20201.
2024/01/09 14:15:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20200.
2024/01/09 14:11:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20199.
2024/01/09 14:08:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20198.
2024/01/09 14:03:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20197.
2024/01/09 13:59:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20196.
2024/01/09 13:55:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20195.
2024/01/09 13:51:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20194.
2024/01/09 13:47:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20193.
2024/01/09 13:43:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20192.
2024/01/09 13:39:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20191.
2024/01/09 13:35:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20190.
2024/01/09 13:32:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20189.
2024/01/09 13:27:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20188.
2024/01/09 13:23:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20187.
2024/01/09 13:19:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20186.
2024/01/09 13:16:00 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20185.
2024/01/09 13:11:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20184.
2024/01/09 13:07:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20183.
2024/01/09 13:07:05 info     url-fil        url-bac 0  Backup of PAN-DB finished successfully.
2024/01/09 13:03:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20182.
2024/01/09 12:59:59 info     url-fil        upgrade 0  PAN-DB was upgraded to version 20240109.20181.

After the CLI command "debug device-server pan-url-db cloud-reelect" it sees that the cloud version is newer, but it doesn't update the local version:

show url-cloud status

PAN-DB URL Filtering
License :                          valid
libcurl resolver :                 threaded
Current cloud server :             serverlist2.urlcloud.paloaltonetworks.com
Cloud connection :                 connected
Cloud mode :                       public
URL database version - device :    20231111.20140
URL database version - cloud :     20231215.20216  ( last update time 2023/12/15 15:22:49 )
URL database status :              good
URL protocol version - device :    pan/2.0.0
URL protocol version - cloud :     pan/2.0.0
Protocol compatibility status :    compatible