Best Practices URL Filtering Recommendations
Note: This post was updated on June 27, 2022 to reflect recent changes to Palo Alto Networks' URL Filtering feature.
Learn more about URL Filtering categories, including block recommended, Consider block or alert, and how they differ from default alert in this to-the-point blog post.
Best Practices: URL Filtering Category Recommendations
Our URL categories can be divided into three broad classes for consideration:
- Block Recommended – Categories consist of known threats, allow subversion of controls, or involve tools and methods used by known threats
- Consider Block or Alert – Categories should be considered with regard to each organization’s legal concerns and compliance policies
- Default Alert – Categories consisting of benign content
In this best practice document, I will focus on the “Block Recommended” and “Consider Block or Alert” classes. While the “Default Alert” content is considered benign, organizations should consider which content is appropriate for their audience.
Block Recommended
Known Threats
- Command and control, block: Malicious URL
If you see C2 in your logs, it could be a strong indicator that an endpoint has been compromised and is attempting to reach out. It is recommended to further investigate the endpoint to check for compromise and potential lateral movement.
- Malware, block: Malicious URL
- Phishing, block: Malicious URL
- Ransomware, block: Malicious URL
Subversion - allows subversion of controls
- Proxy avoidance and anonymizers, block: Proxy servers and other methods that bypass URL filtering or monitoring.
If proxy avoidance is allowed, URL Filtering and other Palo Alto Networks related features will not have visibility into this encrypted traffic. Thus, we cannot block access to malicious URLs, downloads, etc.
Threat Adjacent – Involves tools and methods used by known threats
- Dynamic DNS, block: Websites that provide and/or utilize dynamic DNS services to associate domain names to dynamic IP addresses. Dynamic DNS is often used by attackers for command-and-control communication and other malicious purposes. Many malicious websites are resolved by DDNS; therefore, we recommend blocking this category. If there are specific DDNS websites that need to be accessed, we recommend specifically putting those in a custom category to allow access.
- Grayware, block: Websites that do not pose a direct security threat but do display obtrusive behavior.
These could include websites that run unauthorized crypto-mining javascript on your browser (not persistent, quit the browser and issue goes away), hijackers that change the elements of the browser, such as the default landing page, default search engine, and installing an unauthorized extension. While these behaviors are generally easily reversed, it is not desirable, and we recommend this category to be blocked.
- Hacking, block: Websites relating to the illegal or questionable access to or the use of communications equipment/software. Development and distribution of programs, how-to advice and/or tips that may result in the compromise of networks and systems. This also includes websites that facilitate the bypass of licensing and digital rights systems. Generally, there is no reason to allow end-users to connect to these websites. For organizations that have threat research, SOCs, etc., you may choose to allow access to this category based on user functionality. At the same time, we recommend for those with access, that the action be set to “alert” for logging purposes.
Consider Block or Alert
Legal / Policy
- Adult, block: These websites may be inappropriate in the workplace, school, etc.
- Extremism, block: These websites promote terrorism, racism, facism, or other extremist views discriminating people or groups of different ethnic backgrounds, religions or other beliefs. Some content from extremism websites are highly graphic.
- Abused drugs, block: These websites promote the abuse of both legal and illegal drugs, use and sale of drug related paraphernalia, manufacturing and/or selling of drugs.
- Weapons, block: Sales, reviews, descriptions of or instructions regarding weapons and their use. These websites may not be appropriate in a professional or educational setting.
- Copyright infringement, block: Web pages and services that are dedicated to illegally offering videos, movies, or other media for download infringing copyrights of others. End-users accessing copyright material may expose the organization to copyright violations/penalties.
- Cryptocurrency, alert: Websites that promote crypto-currencies, crypto-mining websites (but not embedded crypto-miners), crypto-currency exchanges and vendors, and websites that manage crypto-currency wallets and ledgers. This category does not include traditional financial services websites that reference crypto-currencies, websites that explain and describe how crypto-currencies and block chains work, or websites that contain embedded crypto-currency miners (grayware).
- Peer to peer, block: Websites that provide access to or clients for peer-to-peer sharing of torrents, download programs, media files, or other software applications. This is primarily for those sites that provide bitTorrent download capabilities. Does not include shareware or freeware websites.
- Alcohol and tobacco, alert: Websites that pertain to the sale, manufacturing, or use of alcohol and/or tobacco products and related paraphernalia. The content of these websites may not be appropriate depending on the audience (i.e., K-12 would block this category.)
- Gambling, block: Lottery or gambling websites that facilitate the exchange of real and/or virtual money. Also related websites that provide information, tutorials, or advice regarding gambling, including betting odds and pools.
- Questionable, block: Websites containing tasteless humor, offensive content targeting specific demographics of individuals or groups of people, criminal activity, illegal activity, and get rich quick websites.
Unknown
- Newly Registered Domains, block (default is alert): Domains that have been newly registered or had a change in ownership within the last 32-days. There is a strong correlation between newly registered domains and malicious URLs.
- Parked, block (default is allow): URLs that host limited content or click-through ads which may generate revenue for the host entity but generally do not contain content that is useful to the end user.
- Unknown, block (default is allow): Unknowns are URLs that URL Filtering has never visited to categorize the content of the page. Recommendation is to block, but many customers are wary of workflow disruption. If a block is not possible, it is recommended to alert, set a much stricter threat prevention profile, and block downloads of dangerous files (PEs, powershells, etc.). Evidence has shown that many customers run into issues with unknowns by allowing end users to download items from these websites.
- Insufficient Content, block (default is allow): Websites and services that present test pages, no content, provide API access not intended for end-user display, or require authentication without displaying any other content suggesting a different categorization.
- Not-resolved, block (default is allow): This is a failure condition where a connection cannot be established between the firewall and the cloud. Not-resolved indicates that the website was not found in the local URL filtering cache, and the firewall was unable to connect to the cloud database to check the URL’s category. Consider failing-closed (block), rather than failing-open (alert).
Risk
- High Risk, alert: Websites that were previously confirmed to be malicious but have displayed benign activity for at least 30 days. Websites hosted on bulletproof ISPs or using an IP from an ASN that has known malicious content. Websites sharing a domain with a known malicious website. All websites in the “Unknown” category will be high risk.
- It is recommended to alert, enforce SSL decryption for increased visibility, set a much stricter threat prevention profile, and block downloads of dangerous files (PEs, PowerShell, etc.) from high risk sites. In addition, you may wish to increase logging for additional insight.
- Medium Risk, alert: Websites confirmed to be malicious but have displayed benign activity for at least 60 days. All websites in the “Online Storage and Backup” category will be medium risk by default.
- It is recommended to alert, enforce SSL decryption for increased visibility, set a much stricter threat prevention profile, and block downloads of dangerous files (PEs, PowerShell, etc) from medium risk sites. In addition, you may wish to increase logging for added insight.
- Low Risk, alert: The low risk category includes websites that have a history of only benign activity and websites that have been found to be malicious in the past but that have displayed benign activity for at least 90 days.
