Categorisation mismatch Test url categorises but debug dataplane url resolve path doesn't

[uduwawalan]

Hi,

Equipment: PA3020, Software: 6.0.2,  Filtering Birghtcloud

We have a situation where large numbers of web sites are being categorised as "unknown.". Upon investigation ( case has now been raised with PA), we find that the Test url <url> command will get the correct categorisation. However the user trying to use the same url will be blocked because their session gets the  url categorised as "unknown". You can replicate this behaviour by running debug dataplane test url-resolve-path <url> which gets you an "unknown" category.

Examples are given below taken out of a console session to our box.

admin@PA-3020> test url dashboard.groupcall.com

dashboard.groupcall.com business-and-economy (Dynamic db)  ----- This is the same categorisation on Brightcloud data base if you visit that web site and do a manual category check.

admin@PA-3020> debug dataplane test url-resolve-path dashboard.groupcall.com

URL dashboard.groupcall.com/, category unknown

admin@PA-3020> test url misguidedchildren.com

misguidedchildren.com games (Dynamic db)

admin@PA-3020> debug dataplane test url-resolve-path  misguidedchildren.com

URL misguidedchildren.com/, category unknown

So.. any ideas please ? I would lke to investigate this issue and get to the bottom of it. It's affecting our day to day operations and ruining the web experience of a lot students and staff.

Thanks for any help.

[mizhou]

Have you tried to do a 'clear url-cache all'?

This will clear the cache on the DP for the BC URL filtering.

You may refer to below tech note for more info:

How to Clear URL-cache using BrightCloud for URL filtering

[uduwawalan]

HI Mzh,

Thanks. Unfortunately that and other clear db statements were the first to tried along with much more aggressive deletion of the DB. The problem re-appears. This is now being investigated by PA. From my understanding, some brightcloud servers across the world don't seem to synchronise their databases reliably all the time.

This leads to a situation where on some occasions our box might be forwarding the url category request to a Brightcloud server that doesn't necessarily have all the category  data! We then get "unknown" as the result for our query. This is a tentative conclusion and Palalto have asked us to do some wireshark captures to be more conclusive.

So for now it's a waiting game.

[pwoll]

I have seen the same behavior, and it appears to still be unresolved.  Categorization changes sometimes take several days to propagate through Brightcloud's infrastructure.  Was there ever any projected resolution from PAN on this?

[ssharma]

Hi  uduwawalan,

Can you run "show system resources | match srvr" and paste the output. Also, is your management plane traffic going through the firewall or does it has its own internet access?

If possible, can you restart both management server and device server.

debug software restart management-server

debug software restart device-server

Run the same test again and see if you notice any difference. Note that above commands will not cause traffic outages. If you are concerned, you can schedule a window to perform the test. Hope this helps. Thank you.