URL filtering database updates problem

adminglu — Tue, 28 Nov 2023 10:28:14 GMT

I have a feeling that the URL filtering database updating doesn't work correctly. After a reboot of the firewall it update a few days and then stops. In the URL filtering logs I also see a lot of "not-resolved" even for url's like play.google.com. I've had this with at least PAN-OS 10.2.4-h2 and now also with 11.0.2-h2. When I run the command "show url-cloud status" it seems to be working, accept the last update time:

PAN-DB URL Filtering License : valid libcurl resolver : threaded Current cloud server : serverlist2.urlcloud.paloaltonetworks.com Cloud connection : connected Cloud mode : public URL database version - device : 20231111.20140 URL database version - cloud : 20231111.20140 ( last update time 2023/11/11 10:16:12 ) URL database status : good URL protocol version - device : pan/2.0.0 URL protocol version - cloud : pan/2.0.0 Protocol compatibility status : compatible

The management interface can communicate with the Palo Alto servers and I see no blocked traffic. Has anybody seen similar issues and is there a service I can restart to get it working again? Rebooting the firewall isn't realy an option.

Thanks in advance.

Re: URL filtering database updates problem

aleksandar.astardzhiev — Thu, 21 Dec 2023 11:38:05 GMT

Hi @adminglu ,

10.2 and 11.0 are fairly new versions any you maybe hitting some known or unknown bug.

It will be better to submit a ticket to TAC to have a look.

Re: URL filtering database updates problem

Adrian_Jensen — Tue, 26 Dec 2023 23:11:18 GMT

I have seen 2 issues that match the description.

I have seen the URL database seeming reset and have to re-initialize, which may take several minutes.
There is a known issue specifically with Google search URLs (including play.google.com/) and URL filtering. Apparently a change was made that overwhelmed the URL cloud database. PaloAlto put in a workaround but it seems that sometimes that workaround misses or they are pushing updates which temporarily cause not-resolved again. Have seen the issue intermittently for months.

Re: URL filtering database updates problem

adminglu — Tue, 09 Jan 2024 14:21:01 GMT

Hi @aleksandar.astardzhiev ,

We have a ticket with our support parter, but so far there is no solution of a root cause found. If there is no solution soon I will have them excalate it to TAC.

Re: URL filtering database updates problem

adminglu — Tue, 09 Jan 2024 14:34:06 GMT

We have now deleted the license, then rebooted the firewall and then re-applied the license. We will now continue to watch the URL-DB version to see if it keeps updating. When it is broken we only see URL DB Backup events, and after a reboot we also see URL DB update events every few minutes. This is the log when broken:

show log system direction equal backward receive_time in last-7-days | match PAN-DB 2023/12/15 12:58:48 info url-fil url-bac 0 Backup of PAN-DB finished successfully. 2023/12/15 08:58:42 info url-fil url-bac 0 Backup of PAN-DB finished successfully. 2023/12/15 04:58:36 info url-fil url-bac 0 Backup of PAN-DB finished successfully. 2023/12/15 00:58:31 info url-fil url-bac 0 Backup of PAN-DB finished successfully. 2023/12/14 20:58:25 info url-fil url-bac 0 Backup of PAN-DB finished successfully. 2023/12/14 16:58:19 info url-fil url-bac 0 Backup of PAN-DB finished successfully. 2023/12/14 12:58:13 info url-fil url-bac 0 Backup of PAN-DB finished successfully.

And this is after a reboot:

show log system direction equal backward receive_time in last-7-days | match PAN-DB 2024/01/09 15:28:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20218. 2024/01/09 15:24:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20217. 2024/01/09 15:20:01 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20216. 2024/01/09 15:16:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20215. 2024/01/09 15:12:02 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20214. 2024/01/09 15:08:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20213. 2024/01/09 15:04:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20212. 2024/01/09 15:00:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20211. 2024/01/09 14:56:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20210. 2024/01/09 14:52:01 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20209. 2024/01/09 14:48:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20208. 2024/01/09 14:44:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20207. 2024/01/09 14:40:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20206. 2024/01/09 14:36:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20205. 2024/01/09 14:32:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20204. 2024/01/09 14:27:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20203. 2024/01/09 14:24:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20202. 2024/01/09 14:19:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20201. 2024/01/09 14:15:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20200. 2024/01/09 14:11:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20199. 2024/01/09 14:08:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20198. 2024/01/09 14:03:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20197. 2024/01/09 13:59:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20196. 2024/01/09 13:55:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20195. 2024/01/09 13:51:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20194. 2024/01/09 13:47:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20193. 2024/01/09 13:43:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20192. 2024/01/09 13:39:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20191. 2024/01/09 13:35:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20190. 2024/01/09 13:32:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20189. 2024/01/09 13:27:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20188. 2024/01/09 13:23:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20187. 2024/01/09 13:19:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20186. 2024/01/09 13:16:00 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20185. 2024/01/09 13:11:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20184. 2024/01/09 13:07:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20183. 2024/01/09 13:07:05 info url-fil url-bac 0 Backup of PAN-DB finished successfully. 2024/01/09 13:03:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20182. 2024/01/09 12:59:59 info url-fil upgrade 0 PAN-DB was upgraded to version 20240109.20181.

After the CLI command "debug device-server pan-url-db cloud-reelect" it sees that the cloud version is newer, but it doesn't update the local version:

show url-cloud status PAN-DB URL Filtering License : valid libcurl resolver : threaded Current cloud server : serverlist2.urlcloud.paloaltonetworks.com Cloud connection : connected Cloud mode : public URL database version - device : 20231111.20140 URL database version - cloud : 20231215.20216 ( last update time 2023/12/15 15:22:49 ) URL database status : good URL protocol version - device : pan/2.0.0 URL protocol version - cloud : pan/2.0.0 Protocol compatibility status : compatible

topic Re: URL filtering database updates problem in Next-Generation Firewall Discussions

URL filtering database updates problem

Re: URL filtering database updates problem

Re: URL filtering database updates problem

Re: URL filtering database updates problem

Re: URL filtering database updates problem