BrightCloud URL-Filter is rubbish

Reply
Highlighted
L3 Networker

BrightCloud URL-Filter is rubbish

We just have declared to block the category "spam-urls" on the firewall.

Since then, a lot of normal german URLs is blocked wrongly. For instance www.ske.de (banking) or www.paderborn.com (official city-website).

I am wondering, how a website could be a email spam category. Or do i think wrong?

mfg

Manfred

Highlighted
Not applicable

Re: BrightCloud URL-Filter is rubbish

Have you tried submitting the URL and/or domains to Brightcloud to have the research the sites and potentially re-categorize?  The also ask what you believe the site should be classifed as.

I do wonder how long the process takes and also wonder why the URL pattern database is only retrieved once a day (unlike the AV and Threats that are updated once an hour).

BF

Highlighted
L3 Networker

Re: BrightCloud URL-Filter is rubbish

Yes i send some hints to Brightcloud. But after a while i removed the block rule because of the large number of Websites in this category. I guess, the category is for Email-Application-Rules only and not very reliably.

Highlighted
L5 Sessionator

Re: BrightCloud URL-Filter is rubbish

Hello,

BrightCloud classifies URLs as "spam-URL" as URLs that are contained in spam messages.  You can find the full list of category descriptions here:

http://www.brightcloud.com/support/catdescription.php

Please do continue to submit category change requests as they will only help to improve the database.  Should you also have a long list of URLs that you would like to submit changes for, you may also open a ticket with support to submit your entire list.

Thanks,

Doris

Highlighted
L4 Transporter

Re: BrightCloud URL-Filter is rubbish

rferry wrote:

Have you tried submitting the URL and/or domains to Brightcloud to have the research the sites and potentially re-categorize?  The also ask what you believe the site should be classifed as.

I do wonder how long the process takes and also wonder why the URL pattern database is only retrieved once a day (unlike the AV and Threats that are updated once an hour).

BF

I've had to reclassify numerous URL's, and generally the process is completed in 48 hours (24 for them to modify the database and publish, another 2 for it to be applied to the PA in the next scheduled download).

The URL database is only retrieved once a day because as far as I can tell Brightcloud only publish updates once a day - so there's no point in doing it more often.

Cheers.

Highlighted
L0 Member

Re: BrightCloud URL-Filter is rubbish

Hi,

One thing I have noticed with the URL classification is if you dont have the dynamic update box ticked some URL's are misclassified as they are not in the initial cache that it downloads.

This is just my experience, I may be wrong.

Marc

Highlighted
L3 Networker

Re: BrightCloud URL-Filter is rubbish

Every URL Filtering product has false positives. I support Websense for many years and it was a regular occurance. That being said, I would hope that if the database is consistently wrong that PA would look to fund a different one. That's my 2 cents.

Justin

Highlighted
L4 Transporter

Re: BrightCloud URL-Filter is rubbish

We have a big problem with small churches having their websites hosted on the site servers that adult websites, and malware sites are hosted on. I just keep submitting them to Brightcloud.

I think that's the big reason Palo Alto included custom URL categories in PAN-OS 3.1.

Highlighted
L4 Transporter

Re: BrightCloud URL-Filter is rubbish

We have also noticed porn sites having a default web page that says "This is my Church".  We submit the URL to Brightcloud and we use the custom URL categories to add the sites as work-allowed until Brightcloud updates their database. We sometime use the lookup services of other URL filtering services to double check the site. We also use custom url categories for IP based urls that we identify and our DMZ hosts that are accessed using local DNS entries and thus are not FQDNs which result in being categorised as unknown.

Hope this helps.

Phil

Highlighted
L4 Transporter

Re: BrightCloud URL-Filter is rubbish

HITSSEC wrote:

We have also noticed porn sites having a default web page that says "This is my Church".  We submit the URL to Brightcloud and we use the custom URL categories to add the sites as work-allowed until Brightcloud updates their database. We sometime use the lookup services of other URL filtering services to double check the site. We also use custom url categories for IP based urls that we identify and our DMZ hosts that are accessed using local DNS entries and thus are not FQDNs which result in being categorised as unknown.

Hope this helps.

Phil

You don't need to go to the extent of adding a custom URL category to make this work.

There's an option in the URL filtering profile setup labelled "allow list" - if you're 100% sure of the site concerned, or if Brightcloud won't re-classify the URL into a category you think it should be in, just put the URL into this field (sans the http:// or https:// protocol extension) and it'll be allowed through regardless of the brightcloud category it falls into.

I use this one as a quick-fix while I wait for brightcloud to re-classify the URL if the user who needs it is really complaining and doesn't want to wait 48 hours.

Cheers.

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!