Security policy matches traffic, but custom URL category doesn't work

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Security policy matches traffic, but custom URL category doesn't work

L2 Linker

Hello,

 

We're running VM appliance with PAN-OS 10.1.4

I've created custom URL category matching several wcard domain names, that we want to whitelist for only some LDAP users and added security policy placing it above internet access policy for all users (where URL filtering profile is applied to block URLs by predefined categories). We're decrypting this traffic as well. Policy looks like this:

 

Src zone: Inside facing user networks
Src User: user1/user2/user3 (IP addresses obtained via querying PA user agent running in separate VM)
Dst zone: Outside facing Internet
Service: Any
Application: Any
Profile: None
URL Category: Custom URL category I've created
Action: Allow

 

This policy matches traffic generated from user1/2/3, but URLs are still blocked and notification indicates them being blocked by default category. Is there anything I'm missing?

Pushing zeros and ones.
8 REPLIES 8

L6 Presenter

Are the URLs blocked in the default category, but allowed in the custom category? Something I ran into before is that a URL Filtering Block always takes precedence over Allow, even if the Allow URL is more specific. 

L2 Linker

Thanks for the reply.

Yes these URLs matched by custom URL category are blocked in default category assigned to URL filtering profile. But security policy rule, that uses this default category filtering profile is below (after) this specific rule. I've even tried making another URL filtering profile where I allowed this custom category, but even then they were blocked. There is no overlap of regex matches in single URL profile.

 

If block always takes precedence regardless its security rule position, then I can't understand how to allow portion of webpages to specific users/IPs, while blocking them to others. It's either all allow or all block then...

Pushing zeros and ones.

L2 Linker

There is an odd thing I've noticed in logs. If this security rule above doesn't have custom URL category attached to it, matched traffic (by source LDAP user) is correctly allowed and forwarded, but once I attach URL category although it matches the URL regex, firewall continues evaluating security rules and finally blocks URL because it's blocked in rule below for all users.

Pushing zeros and ones.

L6 Presenter

So when you define a URL Category it can be used in 2 places:

 - a Service/URL Category traffic identifier under a Security Policy

 - in a URL Filtering object as an allow/block category (always appears in all URL Filtering objects defaulted to "none")

 

So you are using your custom URL Category in your specific allow Security Policy, but under the Actions tab of the policy where you set Allow, do you have a Profile Setting with a URL Filtering profile that is actually blocking the connection? Or perhaps the site is also matches a built in category that is set to block in the URL Filtering.

 

I.e.:

URL Category -> "Example":

    example.com/

    *.example.com/

 

URL Filtering -> "filterInternet"

    Example: block/block

    adult: block/block

    business: allow/allow

    government: allow/allow

    social-media: block/block

 

Policy -> Security -> "Allow general internet access"

    SrcZone = Trust

    DstZone = Untrust

    Action = Allow

      Profile-URL-Filtering = filterInternet   <= general block of example.com by category

 

Policy -> Security -> "Allow specific users to example.com"

    SrcZone = Trust

    SrcUser = Alice, Bob, Eve

    DstZone = Untrust

    URL Category = Example

    Action = Allow

      Profile-URL-Filtering = filterInternet   <= still blocking of example.com by category

 

So, for example, in our environment we block Facebook/Twitter/etc. by default using the build in social-networking category. We then have several custom URL categories where we define some Facebook/etc. domains and a custom URL Filter with those categories set to allow. Then, for a specific set of users we filter traffic in an addition allow rule using the alternate URL Filter. We don't try to identify the target URLs in the Security Policy itself.

 

Policy -> Security -> "Allow general internet access"

    SrcZone = Trust

    DstZone = Untrust

    Action = Allow

      Profile-URL-Filtering = filterInternet   <= general block of social-networking category

 

Policy -> Security -> "Allow specific users to example.com"

    SrcZone = Trust

    SrcUser = Alice, Bob, Eve

    DstZone = Untrust

    Action = Allow

      Profile-URL-Filtering = filterInternet-allowFacebook   <= general block of social-networking category with additional custom URL Category "Facebook-URLs" set to allow/allow

 

L2 Linker

Hello, thank you again for your time.

So you are using your custom URL Category in your specific allow Security Policy, but under the Actions tab of the policy where you set Allow, do you have a Profile Setting with a URL Filtering profile that is actually blocking the connection?

True, I use rule with URL category in specific allow, but no profile is applied in that rule. Let's assume my URL category object name is "CAT1" and has following regex configured:

*.example.com/*

*.example.com/

.example.com/*

.example.com/

And URL filtering profile "PROF1", that has default categories either allowed or blocked, where "CAT1" is set to none. "example.com " is in one of the blocked default categories here.

Then I have two security rules as follows from up->down

Rule1 that allows several LDAP users to any destinations using any apps and any services, "CAT1" applied to it and profiles set to none.

Rule2 for "General Internet Access", that has profiles enabled and "PROF1" applied to it.


What I think I see in logs is, that CAT1 is matched before PROF1, but instead of allowing this traffic, firewall continues evaluating security policies down the line and finally blocks URL via PROF1.

 

I've also tried the scenario described by you using two separate URL filtering profiles, where in PROF1 CAT1 is set to "none/none" and in PROF2 (clone of PROF1) CAT1 is set to "allow/allow". Even then, firewall still blocks access from Rule2 via PROF1.

Pushing zeros and ones.



What I think I see in logs is, that CAT1 is matched before PROF1, but instead of allowing this traffic, firewall continues evaluating security policies down the line and finally blocks URL via PROF1.

 

I've also tried the scenario described by you using two separate URL filtering profiles, where in PROF1 CAT1 is set to "none/none" and in PROF2 (clone of PROF1) CAT1 is set to "allow/allow". Even then, firewall still blocks access from Rule2 via PROF1.



 Hmmm, yeah it sounds like you are on the right track, but the PA is taking the General Internet Access rule as a more specific match. Other than that you do not need both *.example.com/ and *.example.com/* in the category (doesn't cause a problem, just a duplicate), I don't see a problem and am a bit stumped.

L2 Linker

I found what the problem was. It has to do with application IDs.

 

If you leave application to "any" and apply URL filtering profile with URL category, it only matches "web-browsing" and "ssl" app-ids for URL list regex. Anything else associated with that URL will be governed by predefined category containing it and get blocked. If you add the rest of app-ids exclusively (necessary for a webpage like wetransfer for example), then it will correctly allow traffic on that rule.

 

That's why when I was matching with "any" in application field, I could see hits on specific rule but also see blocking from generic rule. web-browsing and ssl were matched in specific rule, wetransfer-base etc., matched further down in generic. Seems like "any" doesn't really mean any in the world of PA...

 

Actually if URL requires only web-browsing and ssl app-ids it works even with URL category only, but there is a caveat. Rules with URL category only will ignore any other app-ids (even added exclusively) except web-browsing and ssl. Pretty convoluted I should say.

 

 

Thanks for helping me 🙂

Pushing zeros and ones.


@Flang3r wrote:

I found what the problem was. It has to do with application IDs.

 

If you leave application to "any" and apply URL filtering profile with URL category... If you add the rest of app-ids exclusively (necessary for a webpage like wetransfer for example), then it will correctly allow traffic on that rule..

 

[...]

 

Actually if URL requires only web-browsing and ssl app-ids it works even with URL category only, but there is a caveat. Rules with URL category only will ignore any other app-ids (even added exclusively) except web-browsing and ssl. Pretty convoluted I should say.

I know that it's been a year and more since this was written, but I find those two statements to be a contradiction to each other - If I understand this correctly, in the first one, you state that if "add the rest of the app-ids exclusively, then it will correctly allow traffic on that rule", but then in the second paragraph you state that "rules with URL category only will ignore any other app-ids (even added exclusively) " - so which is it and how can those two both be true ? what am I missing here ?


Please mark helpful responses, so others know as well
  • 4923 Views
  • 8 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!