Problem with (URL Category custom), (Destination Address any) and (application any)

[Matheus-Reis]

We have identified for some time that when rules are created with 'Application: Any' + 'Destination Address: Any' + 'UrlCategory: Custom' for example:

Name: Rule_google.com_permit
Source Zone: Trust
Source Address: Any
Source User: Any
Destination Zone: Untrust
Destination Address: Any
Application: Any
Service: 443-tcp
URL Category: URLC_Google.com
URL Filtering: URLF_Google.com_Alert
Action Policy: Allow

Other accesses start to match this rule, but it only identifies up to the destination IP + port. The traffic ends up being allowed when viewed in the 'traffic' logs, but the user's access to a site other than the one allowed in the rule in question ends up not working, as if it did not actually match due to the UrlCategory, since the URL he tried to access was not included in the 'google.com' Custom Category.

It makes sense not to follow this criteria. The problem is that when this happens, in my opinion, they shouldn't have matched this rule, but rather followed the 'security policy lookup' process until they found a rule that either accepted all the criteria used by the user, or it didn't and fell under the 'deny-all' rule.

We have some example rules along these lines and they are all 'action:allow', the rules that have a Custom UrlCategory but are 'action:deny' behave as expected and only discard and match the rule for those that are actually making requests to the URLs that are in the UrlCategory with the 'block/deny' action.

When creating rules, we always choose to look for an 'app-id', but not all URLs have their own 'app-id', and there are cases where we need to create Wildcards that are for all users above the 'block-geolocation-countries' rule and that is when we create the rules by UrlCategory and the FQDN object ends up not being used, and that is when we fall into this scenario.

Could you help me, has anyone gone through or is going through this, what should I do?

[Matheus-Reis]

Hello,

Can anyone who is or has experienced this behavior help me?

[BPry]

@Matheus-Reis,

This is expected behavior and you'll always see it on the first broad rule that a user hits. The firewall need to allow enough traffic across to actually validate the URL to see if it properly matches that rule, if it doesn't transmit enough traffic to validate the URL before the traffic is "done" then this rule did allow the traffic to process. Hopefully that makes sense.

The reason deny rules are working as you'd expect them is that the firewall handles that traffic differently and the rule, being a deny, will only match once the URL has been identified. Allow rules just fundamentally have to work differently and allow enough traffic to pass to ever identify if it should match this traffic or not.

[Matheus-Reis]

@BPry 

Thanks for your answer.

I understand, but your answer raised a second question, as follows:

1. In my case, the firewall is receiving enough and REAL traffic, since it is valid traffic with a destination, for example: "bbc.com", but it is stopping at the rule because there is only "google.com" in the custom urlcategory, and yet it is 'allow' in the traffic log. Is this really normal? I imagine it has something to do with the "6-tuple", perhaps?

2. Is it possible to get around this by creating rules in this model ('app-id: any', 'destination-address:any' and 'urlcategory-custom')? Without this happening?

Note: I thought about creating a custom application with regex so that when I identify a wildcard, for example:
*.google.com
in the "host" of the 'http-req-headers'

Example:

I insert this custom app-id in the rule to allow it. And when it was necessary to block other URLs, I would create the rule by urlcategory, as it already works. What do you think?

[Matheus-Reis]

@BPry ,

Hello,

Did you understand my questions? Can you help me?

[BPry]

@Matheus-Reis,

You shouldn't be seeing URLs matching that rule outside of what you actually account for in your custom category. Please post the exact list members that you have in your custom category, sounds like you have something configured more broadly than it should be.

[CosminM]

Hello @Matheus-Reis

This is expected behavior because your rule it's first evaluated and matched at the end of session setup phase (slow path), but at that moment any criteria based APP-ID or URL's are ignored.

Packet Flow Sequence in PAN-OS

Any traffic using TCP/443 it will generate a HIT COUNT for that security rule and if you activate also Log at session start, then you will see it in Traffic Logs. For the same Session ID you will have in your traffic logs at least 2 records (start + end).