topic Re: custom url category issues in General Topics

custom url category issues

Brinkman — Tue, 23 Jul 2013 16:09:14 GMT

OK, let me start out with I am not using the URL filtering profiles, only trying to setup whitelists for outbound web using the custom URL categories.

So I built a rule that allows my trust zone to go out to the untrust using web-browsing app and the custom url category which contains the URLs that need to go out. When I try the connection I'm getting 503 errors and seeing 2 entries in my traffic log. The first one is a start type that is allowed by the rule with an any in the URL category, the second is a deny that is getting dropped by the deny all cleanup rule at the bottom with a not-resolved URL category. What I'm trying to figure out is why it isn't being allowed by the URL category.

Re: custom url category issues

HULK — Tue, 23 Jul 2013 16:46:47 GMT

Hi,

Could you please verify the category from below mentioned link.

http://www.brightcoud.com/support/lookip.php

Also check the traffic logs ( click into the magnifying glass symbol of the dropped traffic) for more details.

Thanks

Subhankar

Re: custom url category issues

Brinkman — Tue, 23 Jul 2013 17:18:03 GMT

It comes up with a category but I'm not sure how that applies since I'm not using Brightcloud, just trying to use my own custom categories.

Re: custom url category issues

Brinkman — Tue, 23 Jul 2013 17:53:04 GMT

Result of show session ID command (IPs sanitized)

Session 1587225

c2s flow:

source: x.x.71.42 [ProdApp]

dst: y.y.110.122

proto: 6

sport: 38456 dport: 80

state: INIT type: FLOW

src user: unknown

dst user: unknown

s2c flow:

source: y.y.110.122 [Deep Dark Woods]

dst: x.x.71.42

proto: 6

sport: 80 dport: 38456

state: INIT type: FLOW

src user: unknown

dst user: unknown

start time : Tue Jul 23 08:41:06 2013

timeout : 90 sec

total byte count(c2s) : 730

total byte count(s2c) : 66

layer7 packet count(c2s) : 11

layer7 packet count(s2c) : 1

vsys : vsys2

application : web-browsing

session to be logged at end : False

session in session ager : False

session synced from HA peer : False

layer7 processing : completed

URL filtering enabled : True

URL category : not-resolved

session via syn-cookies : False

session terminated on host : False

session traverses tunnel : False

captive portal session : False

ingress interface : ethernet1/4

egress interface : ethernet1/15

session QoS rule : N/A (class 4)

Re: custom url category issues

gwesson — Tue, 23 Jul 2013 17:59:13 GMT

You may want to try checking the category on the firewall itself:

> check url www.example.com

The session output shows that the firewall isn't able to resolve the URL category. The site(s) you added to the custom URL category may not be the full list of domains. If you were to add paloaltonetworks.com to a custom URL category, there are other categories referenced by that page. Things like CDNs (akamai, etc.), site analytic cookies, and similar content may not be displayed if you are only allowing the custom category you created.

Hope this helps,

Greg

Re: custom url category issues

dburns — Tue, 23 Jul 2013 18:03:47 GMT

Going back to your first statement, "OK, let me start out with I am not using the URL filtering profiles." Do you even have a URL filter lic and have you downloaded a database in the past? If not, I believe the custom URL wont work as theres no database to put the custom URL category in.

Thanks,

Dominic

Re: custom url category issues

Brinkman — Tue, 23 Jul 2013 18:04:37 GMT

I don't have the check url command but if I do a test url with the url I get a "No URL database is loaded" response.

Re: custom url category issues

Brinkman — Tue, 23 Jul 2013 18:07:24 GMT

No I don't have a license, I'm starting to wonder if that's part of the issue since I am getting No URL database is loaded responses when trying to do a test url. Is this something I can update once without the licenses since I don't really need the categories. We are in front of an all server environment and really only need to allow a handful of sites out but unfortunately 2 of the sites have one URL each but about 50 servers doing load balancing/failover for them.

Re: custom url category issues

dburns — Tue, 23 Jul 2013 18:47:59 GMT

You should get a free 30day eval of URL filtering with the device. You should be able to see that online in customer portal (My Devices). Once the 30 day is applied go to device tab -> licenses and activate the URL filter, then Dynamic Updates. *Adding the URL database may require a restart.

Dominic

Re: custom url category issues

Brinkman — Tue, 23 Jul 2013 18:55:32 GMT

OK, I see that option in My Devices so I should be able to make that work. I guess my only concern with the trial license is will I be able to tell if something is going to keep working once the trial runs out. Can I remove the license once I update the URL database?

Re: custom url category issues

zarina — Tue, 23 Jul 2013 19:26:38 GMT

You can set the action to allow/deny URLs when the license expires: https://live.paloaltonetworks.com/docs/DOC-4329

Re: custom url category issues

Retired Member — Tue, 23 Jul 2013 21:07:17 GMT

if you will only use your own custom list you don't need a license.That works without license.

Re: custom url category issues

dyang — Tue, 23 Jul 2013 23:52:47 GMT

Hi everyone,

If you are only using custom categories or the allow/block list, you can do this without having a URL filtering license. The "test url" CLI command queries the cloud and the on-device database for a URL category, which means that you must have a URL filtering license in order to use that command. So for Brinkman's case where he's only using the custom category, this CLI command is not applicable.

Brinkman, when you created your custom category, did you also attach it to a URL filtering object and attach that to your security policy? From your description, it sounds like there's no URL filtering profile that's getting applied with the block.

--Doris

Re: custom url category issues

Brinkman — Wed, 24 Jul 2013 20:10:37 GMT

I can't use the profiles because I don't have the license for doing URL filtering, unless I'm missing something.

Re: custom url category issues

dyang — Wed, 24 Jul 2013 20:21:54 GMT

Hi Brinkman,

You can use a URL filtering profile, but you can only use the allow/block list and custom category portion of a profile - you cannot use any of the categories that are provided to you without a URL filtering license.

--Doris

Re: custom url category issues

Blackstone — Wed, 24 Jul 2013 23:49:07 GMT

Since you know what websites you want to specifically allow, you can just add those specific IP's to an Address Group. Then change your first security rule and add that new address group to the list of destinations instead of the any option.

Re: custom url category issues

Brinkman — Thu, 25 Jul 2013 12:42:36 GMT

The problem with that is I have 8-10 sites that each use between 40-50 IP addresses and can have new ones added as the load increases on them, so I would have a situation where everything could work one day and then the next I would start to see random blocks because they added a new server that isn't in my IP list.

Re: custom url category issues

mr.linus — Thu, 25 Jul 2013 13:57:20 GMT

why not block on fqdn destoination object?

Re: custom url category issues

ukhapre — Fri, 26 Jul 2013 19:28:59 GMT

URL filtering enabled : True

You can remove the profile altogether or adjust URL filters accordingly

URL filtering matches in following order.

Block list

Allow list

Custom

DP cache

MP cache

Cloud