PANDB is not categorising correctly

[uduwawalan]

Hi

Has anyone experienced this problem and found a solution for it ?

Some examples

www.testbase.co.uk  --  Pandb - religion --           clearly wrong            Brightcloud -  Reference and Research  Training and Tools -- correct     

www. michaelwoodside.co.uk  pandb -unknown   clearly wrong            Brightcloud -- Personal sites and Blog     -- correct

Kennelreg.co.uk Pandb - unknown -                     clearly wrong            Brightcloud  -- Business and economy  -- correct

I can go on with examples. What do we do ? we can't be asking to enter url categorisation change requests ? If we were to do that we will be doing that all day at this rate.

we need help URGENTLY please.

Local reseller is not helping with this at all.

Thanks

Nalin.

[dyang]

Hi Nalin,

Thanks for your feedback.  Submitting a change request is the best way to provide feedback on mis-categorizations.  Not only does this allow us to make the change for the specified URL, but it also allows us to take a look from a bigger picture perspective to see how we can correct the classification engine for those types of URLs/category.  As I'm sure you know, you can submit change requests directly from the device, or from our website: Palo Alto Networks URL Filtering - Test A Site This allows you to submit them individually as well as by bulk.  I can see that you've already submitted a change request for kennelreg.co.uk, but I'll go ahead and file one for the other two examples you provided.

That said, in terms of URLs that initially show up as "unknown", please know that any unknown URL automatically triggers a process on our back-end that takes all unknowns and places them in a prioritized queue for crawling and classification.  What this means for you is that while a URL may show up as unknown in your logs, they should get categorized and included in the database shortly.  No action is required on your part, but you are obviously also welcome to submit a change request for these if you want to provide your suggested answer.  In any case, if your device continues to see traffic to this URL, you should see it switch from "unknown" to a known category.

Hope this helps,

Doris

[mbutt]

Following document provides step by step instructions on how to submit category change on PANDB

How to Submit a Mis-Categorized URL for PAN-DB

Let us know if this helps.

Thanks

Numan

[uduwawalan]

HI Doris / Numan,

Thanks for your input. In a situation where hundreds of pages are getting categorised as unknown it's not feasible for us to ask for categorisation. This is a school with a potential 2000 people trying to browse and getting blocked. Their whole experience is ruined. The whole point of the web and using PALO ALTO at great cost is that you get to where you want when you want on the web safely and securely for our students and staff. Anything less is not ideal and is a potential waste of time.

Since posting this request / question, we realised that these same website that are not categorised are categorised largely correctly on Brighcloud and we have switched to brightcloud on a temporary license. Having switched to bright cloud we find that there is further problem where brightcloud categorises a website correctly ( as in Test URL command) but doesn't get translated in to the same category for the user ( as in debug dataplane tes url-resolve-path <url>   command).

This has dented our confidence very badly and is affecting school activity quite badly.

We need a solution to this. From where it's going to come and when is anybody's guess at this minute.

Thanks

Nalin.

[dyang]

Hi Nalin,

As mentioned in my response, URLs that are initially categorized as unknown in PAN-DB should be categorized within a couple of days, as there is an automatic process that goes through all unknowns that hit our server.  As you've noticed, using BrightCloud may have its own set of problems, but if you still have a valid PAN-DB license, I would suggest you take a look and see if your unknown URLs eventually do get assigned a category. 

Based on your feedback, I'm assuming that you are blocking the "unknown" category.  If this is done to minimize security issues, I have seen some customers allow (alert) on this category, but prevent any file downloads.  Similarly, you can also allow (alert) unknowns, unless they are from a certain country.  You should obviously decide what is best for your network, but perhaps others on this board can chime in on what their strategy is for dealing with unknowns.

--Doris