URL Filtering with Multiple Exceptions

[hophead84]

Hi All

We are currently looking to replace our old Bluecoat proxy with Palo Alto URL filtering but having some issues planning out a reliable and sustinable policy. 

For the sake of simplicity, I'll outline a generic scenario and would appreciate some input on how to acheive this. 

Scenario details:

All user will need internet access. By default all users will have access to all URL categories EXCEPT for the Adult,  Malware, Hacking, Gambling, Online-Storage and nudity categories. A URL Filtering policy called "Base URL Filtering Policy" is created, blocking these categories. All other categories are set to "Alert". 

Some more information that may be relevant. 

• PAN-DB URL Filtering is being used
• UserID is setup and working
• SSL Decryption is NOT enabled and it is not practical to enable

The default access security policy which will apply to anyone without any exceptions configured will be the last policy that could be matched. This will conisist of a security policy including:

• Source Zone (Trusted)
• User (Everyone)
• Destination (Internet Zone)
• Application: A list of permitted applications. (NOTE: I have found that if Applications is set to "any", the URL Filtering Profile on the policy isn't enough to stop other random traffic being allowed by this rule. - is this expected? Not sure of best approach here as I really don't want to have to effictivly whitelist every app)   
• Service/URL Category: "Application default" and NO URL category specified
• Actions: Allow, "Base URL Filtering Policy" Profile Specified in "Profile Settings". 

In addition to the standard level of access:

• Users Alice, Bob and Charlie need access to the Hacking URL category
• User Dennis, Edison and Fiona need access to the Gambling category 
• Users Greg and Hannah need access to both Hacking and Online Storage

How is this best acheived?

Two options come to mind: 

Option 1:

I create 3 new Custom URL Filtering policies, allowing what is specified in the "Base URL Filtering Policy"+ the additional categorie(s), and create 3 new security policies for these users?  My concern around this approach is:

• When we make changes to the default access level, we need to modify all these custom policies. 
• As the security policy is only matched by "User", there will need to be a new URL profile and security policy created for every combination of access required. In our actual envrionment, these will get numerous. eg. If Fiona needs Online-Storage in addition to her current access, a 4th set of policy needs to be created for the combination of Gabling and Online Storage. 

Option 2:

In addition to "User", specify a URL Category to match in the "Service/URL Catrgory" section of the security policy.

This seems like the most ideal option. It would allow a single policy to be setup for each exception category, because it should only ever match the rule when the specified category matches. It would allow the same user to be specified on multiple exceptions, without creating a custom "combination" policy if required.  If nothing matches, the last rule applying the base policy for everyone is used. Sounds great, but... 

I have some concerns and questions about its use however.

• There is inconsistant information from PA documentation about its use: Example: 1) https://live.paloaltonetworks.com/t5/Learning-Articles/Security-policy-fundamentals/ta-p/53016#A8 says you MUST allow the web-browsing application as part of the policy. This is not mentioned anywhere else  .                                       Example 2)  https://www.paloaltonetworks.com/documentation/61/pan-os/pan-os/policy/components-of-a-security-poli... says "To block or allow traffic based on URL category, you must apply a URL Filtering profile to the security policy rules. Define the URL Category as Any and attach a URL Filtering profile to the security policy."  - this makes no sense, why would you set the URL Category as "Any" if you want to use it to match a rule in relation to URL filtering?

• This leavs me asking - should this be used at all? Should it be combined with a custom URL Filtering Profile which only allows the category specified in the Service/URL Category tab? How should it be combined with applications?   

My other concern around this stems from a result of some testing I conducted. 

I setup a default browsing policy as described above, and create a new rule above it as described below:

"Allow Gambling" {
profile-setting {
profiles {
url-filtering "Allow Gambling";
}
}
target {
devices {
000000000
}
negate no;
}
to Internet;
from Trust
source 192.168.204.2;
destination any;
source-user testuser;
category gambling;
application [ ssl web-browsing];
service application-default;
hip-profiles any;
action allow;
}

The URL Filtering Policy "Allow Gambling" specified above has the "gambling" category set to alert, all other categories deny.

Test 1: Browse to gambling website https://www.mrgreen.com.

Result: Website opens fine. Traffic Logs show the traffic to the mrgreen.com web server matched the rule "Allow Gambling". URL filtering logs indicate the site was categorised as "gambling". Looks good!

Test 2: Browse to https://www.bet365.com

Result: Website is blocked. Traffic Logs show the traffic to the bet365.com web server matched the rule "Allow Internet". HOWEVER the URL filtering logs indicate the site was still categorised as "gambling". For some reason it did not match the "Allow Gambling" rule. 

--EDIT-------------------------------

Further testing has shown that Bet365 was not matching because it was being identified as the Bet365 application, when the "Allow Gambling" policy only allowed web-browsing and ssl. This is reassuing, But does this mean I need to specify within the policy every single application associated with the "gambling" URL category? As mentioned in the above article, "any" will not suffice, at least "web-browsing" needs to be defined in a policy where a URL Category is defined... ? 

----------------------------------------

In additon to the above, reviewing the traffic logs with a filter to show all log entries matching the rule "Allow Gambling" - there are hundreds of incomplete sessions to multiple destination IPs matching this rule, all showing  a "URL Category" of "any" and no associated URL Filtering log entry. 

As a result of the above, I'm skeptical if this can provide a reliable approach to URL category exceptions to the default policy.

I understand I've covered a lot of different areas here and it might be difficlt to answer them all, but if anyone has time to have a look at this and share their experience or knowlegde in this area it'd be much appreciated.

Cheers

[santonic]

Yeah, it's hard to know what will be reocgnised as an application and what only as a web-browsing to a specific URL like you ecperienced with your 'gambling' testing.

I believe both of your 2 options would work with some additional configuration. In rules where you are allowing web browsking and applying URL filtering options, set application to 'any' and limit service to port 80 and 443 (and maybe 8080). But also make an additional rule above these web browsing rules. In this rule block all unwanted application, preferably with application filters (gamin, remote access, file sharing...).

Maybe it's not ideal solution but it should work. I'm also interested to hear how others are combining URL filtering and application policy for web browsking traffic.

[OtakarKlier]

Hello All,

I've been through this a few times and the first thing we do is get management to agree to an acceptable level of exceptions/deviations from the 'Base/Everyone' policy. This way you can limit the number of polices that are required. However as you pointed out, there will be the need for 'Bob' to get to a specific category that someone else might not need to get to. In this case I have created special rules for either the category or the application that is detected.

I.E. https://www.khanacademy.org/ is educational, however it requires a specific application other than web-browsing or ssl. So in this case I would either have to get managment to agree to allow everyone to be allowed to use the application or create a special rule to only allow certian AD users or groups access.

Yes I agree this can get very tedious and mind bending at times, but I try and keep to the KISS principle and give management the two options and let them choose.

Hope this helps.

[hophead84]

Thanks for your input. I'm working through a solution now and will share my result once complete.