Problem with URL Filtering Order

[DerarAbubaker]

Hello All,

I need a help in my case, i have a bunch of urls which must be permitted and any thing else must be denied. so i create a white list url using the (Custom Objects - URL Filtering) and then i create a URL Security Profile and blocked every categories and just alert my URL Filtering white list and then add it to my (Inside to Outside) policy.

The problem is here when i monitor the url filtering i found some of my urls are blocked beacuse it is exist in another categories like (Computer-and-Internet-info),  i think because of URL filtering order which describe that the order first for the blocked list then the allow list then the custom categories.

Any one face this problem before and if there any tricks to jumb over the order and mke this happen.

I tried to add my custom object directly to my policy and not to use the profile but in this case i can't know the url which will be blocked and in some times i need this to monitor my urls and to know if i need to open some more urls related to our network activity.

[Abdul_Razaq]

Hi @DerarAbubaker , do you have the same URLs configured in different custom URL catogaries, if so, please not that block have more preference than alert.

Please check if below document helps,

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClsmCAC

Please note that whatever url catogary you directly add to policy is a matching condition, it may not address your requirment

[DerarAbubaker]

Hi @Abdul_Razaq  i have just one custom URL category but the conflicts happen between my custom category which have my white listed URLs and with the predifiened categories like ( Computer-and-Internet-info ) , i need to block all the predifiened categories and just permit my custom URLs to allow.

[Abdul_Razaq]

Hi @DerarAbubaker ,

Are you trying to allow any subdomains?. (eg. www.xyz.com/abc) ?)

For this granularity, you need to have decryption.

[DerarAbubaker]

Hi @Abdul_Razaq thanks for your help, actually no because i have many subdomains for this domain and it will be many others in future so it is not a solution could i use even if its working. What about the decryption what do you mean of this? What should i do? and How could decryption help to solve this issue?

Another question how Paloalto firewall decide that this spacific non-public URL belong to that category?

[jbworley]

Just wanted to post on this subject since I appears that PAN has updated the behavior of precedence in URL Filtering Profiles.

On systems running 8.X code, it does appear that the order of precendence follows what is outlined in https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClsmCAC, however, I am running 9.0.6 on my PA-220, and URL Filtering appears to have an additional factor in assigning priority...the source of the list (custom, EDL, pre-defined).

I ran a test of this in my lab.  In this example, I'm browsing to "jack.com".  You can see the URL is ultimately categorized as "streaming-media", however there is a list that shows all lists that this URL is a member of...interesting to note, it shows my EDL "URL", but I have this set to "none" in the filter profile, so I would have expected it to not show up, just the same as my custom URL categories, but it's there.

I have a custom URL category and EDL that both have entry "jack.com"

In my URL filtering profile, I have my custom URL category set to "none".

I have my EDL set to "none".

Per the logs, I know the pre-defined categories of "streaming-media" and "low-risk" were listed, so those are set to "alert".

First, to test the effect of "block" versus "alert" in the same category group, I change "low-risk" to block to see if it will take precedence over "streaming-media".

It does, as the category is now "low-risk" and the action is "block-url".

So, with "low-risk" still set to block, I update my EDL entry to "alert" and test traffic again.

Sure enough, the EDL alert took priority over the pre-defined category block.

Of course, if I set the EDL to "block", it's still the matching category and does block as expected.

Now, with the pre-defined and EDL categories blocking, I update my custom URL category to "alert".

The traffic is reclassified as the custom category, and the traffic is allowed.

In the end, my URL filter profile looks like this.

Based on this behavior, it does appear that Palo provides a legend of matching priority based on how the groups are listed in the filtering profile (top down: custom, EDL, pre-defined).

I just wanted to point this out since it does appear that with 8.X code, regardless of the category group, a URL matching any "block" category cannot be overriden, regardless of the category group, but with 9.X, it is now possible.  Personally, I like this feature.  If PAN categorizes a web site to a specific category that I happen to be blocking in a URL filtering profile, I can use a custom category "white list" to allow it, avoiding any need to make a new policy with matching criteria and a different content inspection profile.