Security policy matches traffic, but custom URL category doesn't work

[igobejishvili]

Hello,

We're running VM appliance with PAN-OS 10.1.4

I've created custom URL category matching several wcard domain names, that we want to whitelist for only some LDAP users and added security policy placing it above internet access policy for all users (where URL filtering profile is applied to block URLs by predefined categories). We're decrypting this traffic as well. Policy looks like this:

Src zone: Inside facing user networks
Src User: user1/user2/user3 (IP addresses obtained via querying PA user agent running in separate VM)
Dst zone: Outside facing Internet
Service: Any
Application: Any
Profile: None
URL Category: Custom URL category I've created
Action: Allow

This policy matches traffic generated from user1/2/3, but URLs are still blocked and notification indicates them being blocked by default category. Is there anything I'm missing?

[Adrian_Jensen]

Are the URLs blocked in the default category, but allowed in the custom category? Something I ran into before is that a URL Filtering Block always takes precedence over Allow, even if the Allow URL is more specific. 

[igobejishvili]

Thanks for the reply.

Yes these URLs matched by custom URL category are blocked in default category assigned to URL filtering profile. But security policy rule, that uses this default category filtering profile is below (after) this specific rule. I've even tried making another URL filtering profile where I allowed this custom category, but even then they were blocked. There is no overlap of regex matches in single URL profile.

If block always takes precedence regardless its security rule position, then I can't understand how to allow portion of webpages to specific users/IPs, while blocking them to others. It's either all allow or all block then...

[igobejishvili]

There is an odd thing I've noticed in logs. If this security rule above doesn't have custom URL category attached to it, matched traffic (by source LDAP user) is correctly allowed and forwarded, but once I attach URL category although it matches the URL regex, firewall continues evaluating security rules and finally blocks URL because it's blocked in rule below for all users.

[Adrian_Jensen]

So when you define a URL Category it can be used in 2 places:

 - a Service/URL Category traffic identifier under a Security Policy

 - in a URL Filtering object as an allow/block category (always appears in all URL Filtering objects defaulted to "none")

So you are using your custom URL Category in your specific allow Security Policy, but under the Actions tab of the policy where you set Allow, do you have a Profile Setting with a URL Filtering profile that is actually blocking the connection? Or perhaps the site is also matches a built in category that is set to block in the URL Filtering.

I.e.:

URL Category -> "Example":

    example.com/

    *.example.com/

URL Filtering -> "filterInternet"

    Example: block/block

    adult: block/block

    business: allow/allow

    government: allow/allow

    social-media: block/block

Policy -> Security -> "Allow general internet access"

    SrcZone = Trust

    DstZone = Untrust

    Action = Allow

      Profile-URL-Filtering = filterInternet   <= general block of example.com by category

Policy -> Security -> "Allow specific users to example.com"

    SrcZone = Trust

    SrcUser = Alice, Bob, Eve

    DstZone = Untrust

    URL Category = Example

    Action = Allow

      Profile-URL-Filtering = filterInternet   <= still blocking of example.com by category

So, for example, in our environment we block Facebook/Twitter/etc. by default using the build in social-networking category. We then have several custom URL categories where we define some Facebook/etc. domains and a custom URL Filter with those categories set to allow. Then, for a specific set of users we filter traffic in an addition allow rule using the alternate URL Filter. We don't try to identify the target URLs in the Security Policy itself.

Policy -> Security -> "Allow general internet access"

    SrcZone = Trust

    DstZone = Untrust

    Action = Allow

      Profile-URL-Filtering = filterInternet   <= general block of social-networking category

Policy -> Security -> "Allow specific users to example.com"

    SrcZone = Trust

    SrcUser = Alice, Bob, Eve

    DstZone = Untrust

    Action = Allow

      Profile-URL-Filtering = filterInternet-allowFacebook   <= general block of social-networking category with additional custom URL Category "Facebook-URLs" set to allow/allow