URL Filter doesn't work in Deny rule

[SeanBui]

I have 2 rules for IT group: IT_Deny and IT_Allow as in the picture below. I'm using a same profile group for both rules, in profile group I have a URL_filter that block some websites like bbc.com, cnn.com

But when I access bbc/cnn, I get blocked by URL filter in profile group in IT_Allow rule. I don't understand why I don't get blocked by IT_Deny rule, IT_Deny rule still let me access bbc/cnn??

[Hithead]

can you please upload a screenshot of the traffic and url logs? (deny and allow case)

[TranceforLife]

Most likely you are not matching all security policy criteria (s) (e.g destination ip). Not sure if security profiles even processed for "deny" action in the security policies. 

[VinceM]

Hi,

It can be explained because your traffic doesn't match your first rule. On ly diff is your app group ITT, which app are embeded ?

On which criteria do you want to allow or block access to these URL ?? AD member ? list of IP ?

V.

[gwesson]

@VinceM and @TranceforLife are both correct. 

When the first packet arrives, it looks to see if it matches your first rule (IT_Deny). If you're attempting to block sites like bbc.com and cnn.com, the first packet will just be a SYN packet on port 443. That SYN packet is not enough to identify the app as SSL for the app-id engine (even though eventually it will be identified as such). Since that first packet doesn't get denied, it won't match that rule until an app-id shift.

When your first TLS packet arrives at the firewall (Client Hello, likely containing the server name) can now identify that it is cnn/bbc, and deny it based on your URL filtering profile. The traffic was being processed on rule 2 (IT_Allow) at the time that information was available, so that's what it matches in your logs.

If you're set on having a deny policy for this, add the category to the IT_Deny policy. But it won't give the user a nice category deny page, instead it'll just drop the packet. It's generally much better to allow the traffic and use the URL filtering profile to handle what is denied and allowed.

[SeanBui]

First of all, Thank you all for your reply. I provide some more information about my rules

In IT_Deny rule I also block some applications. IT_Deny rule works good blocking these apps

This is my Profile group used for both Deny and Allow rule, using URL_filtering profile called ITT_URL

ITT_URL has a blocked category called Block_URL that contains BBC and CNN

And this is my traffic log and URL_filter log when I accessed bbc and cnn. As you can see, these websites are processed by IT_Allow, they passed IT_Deny rule

[Raido_Rattameister · Accepted solution]

It was mentioned that SYN matches second rule and for that reason block rule does not apply. This is not correct.

Every application shift invokes policy check. If application changes to web-browsing then ruleset is checked again.

Your issue is that Security Profiles are checked only if Security Policy permits traffic.

If policy denies traffic then Security Profiles are not checked.

Good example is that there is no reason to look for viruses in traffic that is blocked anyway.

If you want to block URL categories then you:

- Block it in URL profile and set policy action Allow.

- Add URL category directly into policy (under Service/URL Category tab) and set rule action to Deny

@SeanBui I edited reply a bit.

[SeanBui]

Thank you very much, my mind is clear now ❤️

[TranceforLife]

This is exactly what I thought. In any case, URL category is another matching criteria where security profiles an additional future (s) and only actual when the policy action is "allow".

[ChrisCapra]

Had the same issue, figured out had to block it in the URL Profile and set Rule/Policy to allow. 

[OwenFuller]

I realize this has been solved, but this KB article might be useful to others having a similar problem understanding the packet flow sequence in PANOS:  https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClVHCA0

As stated before, if your rule action is Deny, the additional security policy for URL filtering won't make a difference.  For our default URL blocking rules, we actually have an Allow action with the URL policy doing all the denying.  We do this because it's the only way to get URL filtering logs, and give end users those nice block messages with an explanation and directions for opening a ticket.