Video Tutorial: How to Configure URL Filtering

Video transcript:
This is a Palo Alto Networks Video Tutorial. In today's Video Tutorial I will be talking about "How to configure URL Filtering." This will be the first video of a series talking about URL Filtering. This video is designed to help you better understand and configure URL filtering on PAN-OS 6.1.

We will be covering the following topics in this Video Tutorial, as we need to understand all of the parts that make up URL filtering.

1. What is URL filtering?
2. URL filtering vendors
3. Licensing and updates
4. URL filtering components
5. URL filtering profiles
6. Response pages
7. Order of inspection
8. How to configure URL filtering rules
9. What the logs will look like

Let's start out by talking about:

1. What is URL filtering?
The Palo Alto Networks URL filtering solution is a powerful PAN-OS feature that is used to monitor and control how users access the web over HTTP and HTTPS. This feature can be used to gain complete visibility and control of the traffic that traverses your firewall and will be able to safely enable and control how your users access the web.

 

2. Next, let's look at two URL filtering vendors:

• BrightCloud
• PAN-DB

BrightCloud is a vendor that was used in the past, and is still supported, but no longer the default. PAN-DB is Palo Alto Networks very own URL filtering database, and the default now.

3. Licensing and updates
We also need to ensure that you already have the following in place:

• Make sure that you have a valid URL filtering license for either BrightCloud or PAN-DB.
• Make sure that the dynamic updates has been completed
• URL Filtering license, check on the Device > License screen.

PAN-DB or BrightCloud database is up to date

4. URL filtering components
URL categories rules can contain a URL Category. Each website defined in the URL filtering database is assigned one of approximately 60 different URL categories. There are two ways to make use of URL categorization on the firewall:

• Block or allow traffic based on URL category
• Match traffic based on URL category for policy enforcement

By grouping websites into categories, it makes it easy to define actions based on certain types of websites. In addition to the standard URL categories, there are three additional categories:

• Not-Resolved
• Private-ip-addresses
• Unknown

5. URL Filtering Profile

• URL Filtering Profile Actions:
  • Alert
  • Allow
  • Block (Block page displayed to the user)
  • Continue
  • Override
  • None
• Block and allow lists
• Safe search enforcement
• Container ages
• HTTP header logging

6. URL filtering response pages

• Block (Block page displayed to the user)
• Continue (Continue page displayed to the user)
• Override (Page displayed to enter Override password)
• Safe Search Block Page (if Safe Search is enabled on the firewall, but the client does not have their settings set to ‘strict’)

7. The Order URL Filtering profiles are checked:

• Block List
• Allow List
• Custom URL Categories
• DP URL Cache
• MP URL Cache

8. Now, let's configure URL filtering on your firewall.
How to configure URL filtering rules.

Configure a Passive URL Filtering policy to simply monitor traffic.
The recommended practice for deploying URL filtering in your organization is to first start with a passive URL filtering profile that will alert on most categories. After setting the alert action, you can then monitor user web activity for a few days to determine patterns in web traffic. After doing so, you can then make decisions on the websites and website categories that should be controlled.

Note: The default URL filtering profile is set to allow access to all URL categories except for the following threat-prone categories that are blocked: abused-drugs, adult, gambling, hacking, malware, phishing, questionable, and weapons. As a best practice, when you need a custom URL Filtering profile, clone the default profile rather than creating a new one to preserve these settings.

In the procedure that follows, threat-prone sites will be set to block and the other categories will be set to alert, which will cause all websites traffic to be logged. This may potentially create a large amount of log files, so it is best to do this for initial monitoring purposes to determine the types of websites your users are accessing. After determining the categories that your company approves of, those categories should then be set to allow, which will not generate logs. You can also reduce URL filtering logs by enabling the Log container page only option in the URL Filtering profile, so only the main page that matches the category will be logged, not subsequent pages/categories that may be loaded within the container page. 

Inside the GUI, click on Objects > Security Profiles > URL Filtering.

Create a new URL filtering profile by selecting the default policy, and then click 'Clone' at the bottom of that window. It will create a new URL filtering profile - default-1. Click on that name (default-1) and change the name to URL-Monitoring. Because we are monitoring with this profile, we need to set the action of the categories to "alert." By default, the categories will be listed alphabetically. Because we have retained the threat-prone sites, you will see that the action for some sites is set to "block". This action column is also sortable, which you can click on the word "Action".

You will see how the categories change their order and you will now see "allow" in the Action column. This will order the categories making it easy to see which are different. To select all items in the category list, click the check box to the left of Category. This will highlight all categories. If you need to select a few categories, check the first category, then hold down the shift key and click the last category name. Do not select the check box while using the shift key because this will not work properly. If you select more categories than you wanted to, hold the control key (ctrl) down and click items that should be deselected. On a Mac, do the same using the shift and command keys. You could also just set all categories to alert and manually change the recommended categories back to block, but I find this first way easier to remember which categories are threat-prone. 

To the right of the Action column heading, mouse over and select the down arrow and then select "Set Selected Actions" and choose "alert". Click OK.

Apply the URL filtering profile to the security policy rule(s) that allows web traffic for users.

1. Do this by going to Policies > Security and select the appropriate security policy to modify it.
2. Select the Actions tab and in the Profile Setting section, click the drop-down for URL Filtering and select the new profile.
3. Click OK to save.

 

9. What the logs will look like
Look at logs, see the details inside of Monitor > URL filtering

Please remember, since we alerting or blocking all traffic, we will see it. If it is allowed through a rule and does not alert, we will not see an entry for it in the URL filter logs. By default, the "URL Category" column is not going to be shown.  Please click on the 'down arrow' to the right of any column name then click 'Columns' and then check the mark next to "URL category." This will now show you the URL Category in the security rules, and then should make his much easier to see the URL's in the rules.

That concludes this video tutorial. We hope you enjoyed this video. Thanks for watching.