custom url category issues

[Brinkman]

OK, let me start out with I am not using the URL filtering profiles, only trying to setup whitelists for outbound web using the custom URL categories.

So I built a rule that allows my trust zone to go out to the untrust using web-browsing app and the custom url category which contains the URLs that need to go out.  When I try the connection I'm getting 503 errors and seeing 2 entries in my traffic log.  The first one is a start type that is allowed by the rule with an any in the URL category, the second is a deny that is getting dropped by the deny all cleanup rule at the bottom with a not-resolved URL category.  What I'm trying to figure out is why it isn't being allowed by the URL category.

[HULK]

Hi,

Could you please verify the category from below mentioned link.

http://www.brightcoud.com/support/lookip.php

Also check the traffic logs ( click into the magnifying glass symbol of the dropped traffic)  for more details.

Thanks

Subhankar

[Brinkman]

It comes up with a category but I'm not sure how that applies since I'm not using Brightcloud, just trying to use my own custom categories.

[Brinkman]

Result of show session ID command (IPs sanitized)

Session         1587225

        c2s flow:

                source:      x.x.71.42 [ProdApp]

                dst:         y.y.110.122

                proto:       6

                sport:       38456           dport:      80

                state:       INIT            type:       FLOW

                src user:    unknown

                dst user:    unknown

        s2c flow:

                source:      y.y.110.122 [Deep Dark Woods]

                dst:         x.x.71.42

                proto:       6

                sport:       80              dport:      38456

                state:       INIT            type:       FLOW

                src user:    unknown

                dst user:    unknown

        start time                    : Tue Jul 23 08:41:06 2013

        timeout                       : 90 sec

        total byte count(c2s)         : 730

        total byte count(s2c)         : 66

        layer7 packet count(c2s)      : 11

        layer7 packet count(s2c)      : 1

        vsys                          : vsys2

        application                   : web-browsing

        session to be logged at end   : False

        session in session ager       : False

        session synced from HA peer   : False

        layer7 processing             : completed

        URL filtering enabled         : True

        URL category                  : not-resolved

        session via syn-cookies       : False

        session terminated on host    : False

        session traverses tunnel      : False

        captive portal session        : False

        ingress interface             : ethernet1/4

        egress interface              : ethernet1/15

        session QoS rule              : N/A (class 4)

[gwesson]

You may want to try checking the category on the firewall itself:

> check url www.example.com

The session output shows that the firewall isn't able to resolve the URL category. The site(s) you added to the custom URL category may not be the full list of domains. If you were to add paloaltonetworks.com to a custom URL category, there are other categories referenced by that page. Things like CDNs (akamai, etc.), site analytic cookies, and similar content may not be displayed if you are only allowing the custom category you created.

Hope this helps,

Greg

[dburns]

Going back to your first statement, "OK, let me start out with I am not using the URL filtering profiles." Do you even have a URL filter lic and have you downloaded a database in the past? If not, I believe the custom URL wont work as theres no database to put the custom URL category in.

Thanks,

Dominic

[Brinkman]

I don't have the check url command but if I do a test url with the url I get a "No URL database is loaded" response.

[Brinkman]

No I don't have a license, I'm starting to wonder if that's part of the issue since I am getting No URL database is loaded responses when trying to do a test url.  Is this something I can update once without the licenses since I don't really need the categories.  We are in front of an all server environment and really only need to allow a handful of sites out but unfortunately 2 of the sites have one URL each but about 50 servers doing load balancing/failover for them.

[dburns]

You should get a free 30day eval of URL filtering with the device. You should be able to see that online in customer portal (My Devices). Once the 30 day is applied go to device tab -> licenses and activate the URL filter, then Dynamic Updates. *Adding the URL database may require a restart.

Dominic

[Brinkman]

OK, I see that option in My Devices so I should be able to make that work.  I guess my only concern with the trial license is will I be able to tell if something is going to keep working once the trial runs out.  Can I remove the license once I update the URL database?

[zarina]

You can set the action to allow/deny URLs when the license expires: https://live.paloaltonetworks.com/docs/DOC-4329

[panos]

if you will only use your own custom list you don't need a license.That works without license.

[dyang]

Hi everyone,

If you are only using custom categories or the allow/block list, you can do this without having a URL filtering license.  The "test url" CLI command queries the cloud and the on-device database for a URL category, which means that you must have a URL filtering license in order to use that command.  So for Brinkman's case where he's only using the custom category, this CLI command is not applicable. 

Brinkman, when you created your custom category, did you also attach it to a URL filtering object and attach that to your security policy?  From your description, it sounds like there's no URL filtering profile that's getting applied with the block.

--Doris

[Brinkman]

I can't use the profiles because I don't have the license for doing URL filtering, unless I'm missing something.

[dyang]

Hi Brinkman,

You can use a URL filtering profile, but you can only use the allow/block list and custom category portion of a profile - you cannot use any of the categories that are provided to you without a URL filtering license.

--Doris