WWW vs No WWW

[TNaami]

I submit URLs to BrightCloud regularly (several times a week).  Most URLs once updated to the filter database work fine with or without the www as part of the URL.  Every so often I will come across a situation where a URL won't load.  For instance, http://www.paloaltonetworks.com won't load (says category: unknown) but http://paloaltonetworks.com will load correctly.

Doing a CLI test URL www.paloaltonetworks.com gives:

www.paloaltonetworks.com unknown (Dynamic db)

Doing a CLI test URL paloaltonetworks.com gives:

paloaltonetworks.com computer-and-internet-security (Dynamic db)

Doing a clear url-cache does not help.  After a number of days this will usually resolve itself.  (This can take up to a week so I end up having to add the URL to the exception list and then go back and clean this up later.  More work.)

Now here is where it gets interesting.  I have a TEST Palo Alto firewall of the same hardware, firmware, configuration (different addresses of course), same filter database.  It shows the proper category with and without the www.  Thus I know the database is working correctly.

As stated I've cleared the url cache.  This is not something that happens with MOST newly categorized URLs but it happens often enough to be annoying.  Ideas?

[mikand]

And when this happens and you test both urls at URL/IP lookup over at Webroot, Inc. they both will show up as computer-and-internet-security ?

[TNaami]

As far as I can tell Webroot is classifying them the same with and without the WWW.  In fact Webroot strips the www. off the URL when it is submitted but it still comes up the same.

My production system is an active-passive setup of two 2050s.  They are running 4.0.14 code.  My TEST box is a 2050 running 4.0.14 code and has the configuration from my ACTIVE unit with minor changes. (Different IP addresses, no HA peer.)  Here is a real example.  The URL is one I submitted and was published Wednesday night.

Production FW:

admin@PA-2050-A(active)> test url www.earnosethroatspecialistrockford.com

www.earnosethroatspecialistrockford.com unknown (Dynamic db)

admin@PA-2050-A(active)> test url earnosethroatspecialistrockford.com

earnosethroatspecialistrockford.com health-and-medicine (Dynamic db)

admin@PA-2050-A(active)> request url-filtering download status

Jun 13 01:02:24 URL filtering database version 4112 is already the latest version

Test FW:

admin@PA-2050-C> test url earnosethroatspecialistrockford.com

earnosethroatspecialistrockford.com health-and-medicine (Dynamic db)

admin@PA-2050-C> test url www.earnosethroatspecialistrockford.com

www.earnosethroatspecialistrockford.com health-and-medicine (Dynamic db)

admin@PA-2050-C> request url-filtering download status

Jun 13 04:00:25 URL filtering database version 4112 is already the latest version

[VinceM]

Hi,

No reason for having defferent classification with or without www.

The only thing is maybe your first classification request takes too many time then the palo says "unknown" because no answer. Second one is ok.

You can change this timer with command: set deviceconfig setting ctd wait-to-url

V.

[Gregoux]

by default this timer is set to 5 s

and with dinamic url resolution sometime the firwall need more time to ask to the cloud

but if you upgrade the timerr at 10 s it could be work fine after a reboot of panos software.

set deviceconfig setting ctd wait-to-url 10

request restart software

[TNaami]

I didn't change anything on the firewall.  Wednesday I sent in the classification request and it was published and applied that night.  On Thursday the URL without WWW worked but was unknown with the WWW.  As of Friday morning both worked.  No reboots, no commits, no "administrator" functions applied to the firewall since I last checked it. (Not even the PC was rebooted.)   The auto-update agent changed the database from 4112 to 4113 last night as it should.