topic Re: URL categorization reasoning? in General Topics

URL categorization reasoning?

ajr0 — Thu, 30 Apr 2015 20:17:40 GMT

Someone shared a link with me to a new startup company and I found it had been blocked and listed as malware by a PAN FW -- I am curious if there is a way to determine exactly what occurred that made that site categorized as malware -- and how I can lookup other URL's reason for categorization -- if available.

Thanks,

Re: URL categorization reasoning?

rsriramoju — Thu, 30 Apr 2015 21:36:53 GMT

Hi Aron,

Are you using Bright cloud or Pan db for URL categorization?

ou can manually verify the category of any URL by entering it on the following web page:

http://www.brightcloud.com/tools/url-ip-lookup.php

https://urlfiltering.paloaltonetworks.com/testasite.aspx

If you do not agree with the categorization of an individual URL/website you can submit a recategorization request on this page:

http://brightcloud.com/support/changerequest.php

You can also clear the Url cache using below command

>clear url-cache url www.xyz.com

Also please check the below document for your reference.

https://live.paloaltonetworks.com/docs/DOC-2227

-regards

Rajiv

Re: URL categorization reasoning?

ajr0 — Thu, 30 Apr 2015 21:39:54 GMT

Rajiv,

Thanks, I am specifically looking for the reason why a URL was chosen to be categorized as malware -- for example xyz.com is malware because software hosted on site was found to be distributing malware for command and control or Cryptolocker...

Re: URL categorization reasoning?

rsriramoju — Thu, 30 Apr 2015 21:53:46 GMT

Hi Aron,

I dont think we can provide data on the classification history or how classification happens on the back-end.

If a website was mis-categorized and the best way to find out is look at what other vendors say about it.

You can go to www.virustotal.com and find out what vendors like Kaspersky. Fortinet , BitDefender etc say about the website.

If a majority of vendors classify the website as benign but PAN-DB does not , that means we mis-categorized it. At that point you could submit a request.

How to Submit a Mis-Categorized URL for PAN-DB

-regards

Rajiv

Re: URL categorization reasoning?

gwesson — Thu, 30 Apr 2015 22:00:40 GMT

To add some color to rsriramoju's statement, we don't supply specific details most of the time because that is part of our detection algorithm.

Generally speaking, a domain is categorized as malware when there are malicious files hosted at the domain, a piece of malware makes calls to that domain, or similar actions.

More times than not I've seen a malicious file hosted on a compromised server on the domain. It could be a legitimate domain, but due to some unpatched (or zero-day) vulnerability, a malicious actor has planted malware on a directory reachable publicly. Incidentally, it's also why reputation-based systems can fail to protect you.

If you own the domain and have a Palo Alto Networks support contract, you can open a support ticket to uncover more specifics.

Best regards,

Greg Wesson

Re: URL categorization reasoning?

lewis — Mon, 04 May 2015 17:40:48 GMT

If you go to www.virustotal.com you can look up a url and it will have a date saying when it was last scanned and its rating against AV clients. This does not give you the reasons for its categorization nor is it inline with PA's categorization but sometimes can be beneficial when trying to understand a timeline.

Re: URL categorization reasoning?

ajr0 — Tue, 05 May 2015 16:17:51 GMT

Thanks, everything in virus total said it was clean --> www.neucoin.org.

however PAN marks as Malware -- I have personal no evidence supporting it being malware or clean which is why I wanted to see reasoning from PAN.

is there a way to follow up with PAN to determine reasoning? I do not want to submit a re categorization because I do not know that it is clean.

Re: URL categorization reasoning?

lewis — Tue, 05 May 2015 16:27:22 GMT

I am seeing it categorized as Stock Advice and Tools. Maybe they revisited this one for you.

Re: URL categorization reasoning?

ajr0 — Tue, 05 May 2015 16:44:13 GMT

Yes it must have just changed within past day or two as my logs in PA show it as Malware on 05-04-2015 at 11:37 est.

however, I'm still curious as to why it was listed as malware for a brief period of time 🙂

Re: URL categorization reasoning?

HITSSEC — Thu, 07 May 2015 02:39:17 GMT

ajr13,

The web site could have been hosting malware (thus being categorized as malware). If the site administrator was informed and then removed the malware and patched their server to prevent further malware being placed on the server. The site then get a re-categorization request and since the malware is not there anymore it gets classified as it normally should be. This is how I expect it works. Hope this helps.

Phil