Need a way to feed your own IOC's into MineMeld to be consumed by a Palo Alto Networks firewall? Here's a step-by-step guide.

Table of Contents Introduction. Category Mappings For Moving From Symantec WebFilter to PAN-DB. Roll Out URL Category Enforcement Best Practice URL Filtering Profile. Use URL Categories To Define SSL Decryption Policies. URL Filtering Use Cases. Useful Resources.   Introduction   This document is designed to assist you in migrating your environment from using Symantec Web Filter categories on ProxySG to using URL filtering capabilities in the Palo Alto Networks next-generation firewall enabled by PAN-DB, Palo Alto Networks cloud-based URL categorization service.   The first part of this document contains category mappings to assist you in selecting which PAN-DB URL categories to use. In most cases, there is a one-to-one mapping between the URL categorization commonly used in Symantec Web Filter and the categorization provided by Palo Alto Networks.   The second part of the document contains examples on how to migrate from Symantec Web Filter categories to PAN-DB categories and how to use them in the security policies of the next-generation firewall. During the migration, it is a recommended best practice to configure a URL Filtering profile with all categories set to “alert” in parallel with your web filtering solution. This allows you to run reports in PAN-OS and Proxy-SG to verify policies and category mappings before switching the URL filtering functions over completely to PAN-DB URL categorization on our next generation firewall.   The third part of this document contains usage examples and recommended security best practices when using PAN-DB for URL categorization in the next-generation firewall.     Category Mappings for Moving From Symantec WebFilter to PAN-DB   To start the migration, the first thing we recommend is to review the categories that are blocked by policy with the Symantec WebFilter and map them to the corresponding PAN-DB URL categories.   The Symantec WebFilter Database is organized into 85 URL categories. You can find a complete list and definitions of the categories at this link: https://sitereview.bluecoat.com/category-descriptions.   PAN-DB is organized into more than 65 URL categories. You can find a complete list and definitions of the categories at this link: Complete List of PAN-DB URL Filtering Categories.   Symantec WebFilter offers a service called “Site Review.” The purpose of “Site Review” is to allow Symantec customers to check the current database categorization of WebFilter URLs and report sites that they believe are incorrectly categorized. You can find the "Site Review" at this link: https://sitereview.bluecoat.com/.   PAN-DB URL filter also offers a service called “Test a Site.” The purpose of “Test a Site” is to allow Palo Alto Networks customers to check the current database categorization of PAN-DB URLs and report sites that they believe are incorrectly categorized. You can find "Test a Site" at this link: https://urlfiltering.paloaltonetworks.com/.   The table below will help you with the category mapping exercise.   Symantec Palo Alto Networks Differences Recommendations Abortion Abortion     Adult/Mature Content Adult or Questionable     Alcohol Alcohol and Tobacco     Alternative Spirituality/Belief Religion     Art/Culture Entertainment and Arts     Auctions Auctions     Audio/Video Clips Streaming Media or Music     Brokerage/Trading Stock Advice and Tools or Financial Services     Business/Economy Business and Economy     Charitable Organizations Society     Chat (IM)/SMS Internet Communications and Telephony     Child Pornography Adult     Computer/Information Security Computer and Internet Info or Hacking     Content Servers Content Delivery Networks     Controlled Substances Abused Drugs     Dynamic DNS Host Dynamic-DNS   Best Practice recommendation, Block “dynamic-dns” category E-Card/Invitations Shareware-and-Freeware     Education Educational Institutions     Email Web-based Email     Entertainment Entertainment and Arts     Extreme Extremism   Best Practice recommendation, Block “extremism” category File Storage/Sharing Online Storage and Backup     Financial Services Financial Services     For Kids Society This Symantec category is not a stand-alone category   Gambling Gambling     Games Games     Government/Legal Government     Hacking Hacking     Health Health and Medicine     Humor/Jokes Entertainment and Arts or Questionable     Informational N/A This Symantec category is not a stand-alone category Recommended action: – Use “Test a Site” to find corresponding PAN-DB category for matching websites   – Or Create a Custom URL category and control matching websites Internet Connected Devices Computer and Internet Info There is no one-to-one mapping for this category. This is a subset of “computer-and-internet-info” category Recommended action: – Use “Test a Site” to find corresponding PAN-DB category for matching websites   – Or Create a Custom URL category and control matching websites Internet Telephony Internet Communications and Telephony     Intimate Apparel/Swimsuit Swimsuits and Intimate Apparel     Job Search/Careers Job Search     Malicious Outbound Data/Botnets Command-and-Control   Best Practice recommendation, Block “Command-and-Control” category Malicious Sources/Malnets Malware   Best Practice recommendation, Block “malware” category Marijuana Abused Drugs     Media Sharing Streaming Media or Online Storage and Backup     Military Military     Mixed Content/ Potentially Adult Adult, Nudity or Questionable Based on the category description provided by Symantec, most URLs should be mapped to “adult” but the URLs could also be part of “nudity” or “questionable”   News/Media News     Newsgroups/Forums News or Personal-Sites-And-Blogs     Non-Viewable/Infrastructure Insufficient Content     Nudity Nudity     Office/Business Applications Computer and Internet Info     Online Meetings Internet Communications and Telephony     Peer-to-Peer (P2P) Peer-to-Peer     Personals/Dating Dating     Personal Sites Personal Sites and Blogs     Phishing Phishing   Best Practice recommendation, Block “phishing” category Piracy/Copyright Concerns Copyright-Infringement   Best Practice recommendation, Block “copyright-infringement“ category Placeholders Parked   Best Practice recommendation, Block “parked” category Political/Social Advocacy Philosophy and Political Advocacy     Pornography Adult     Potentially Unwanted Software Shareware and Freeware or Questionable     Proxy Avoidance Proxy Avoidance and Anonymizers   Best Practice recommendation, Block “proxy-avoidance-and-anonymizers” category Radio/Audio Streams Streaming Media     Real Estate Real Estate     Reference Reference and Research     Religion Religion     Remote Access Tools Internet Communications and Telephony       Restaurants/Dining/Food Society     Scam/Questionable/Illegal Questionable     Search Engines/Portals Search Engines     Sex Education Sex Education     Sexual Expression Adult or Society If the website content pertains to sexual identity then the category will be "society" If not, the category will be "adult"   Shopping Shopping     Social Networking Social Networking     Society/Daily Living Society     Software Downloads Shareware and Freeware or Computer and Internet Info     Spam Questionable URLs related to spam are included in the category "questionable" This category also includes websites with illegal, immoral and offensive content   Sports/Recreation Sports     Suspicious Insufficient Content or Questionable     Technology/Internet Computer and Internet Info     Tobacco Alcohol and Tobacco     Translation Translation     Travel Travel     TV/Video Streams Streaming Media     Uncategorized Unknown   Best Practice recommendation, Block “unknown” category Vehicles Motor Vehicles     Violence/Hate/Racism Extremism   Best Practice recommendation, Block “extremism” category Weapons Weapons     Web Ads/Analytics Web Advertisements     Web Hosting Web Hosting         Roll Out URL Category Enforcement   The recommended practice for deploying URL filtering in your organization is to start with a “passive" URL Filtering profile that will create log entries by employing the “alert” policy action on all categories in parallel with your existing web filter appliance. On the Palo Alto Networks firewall, create a new URL Filtering profile. Select Objects > Security Profiles > URL Filtering. Select the default profile and then click Clone. The new profile will be named default-1. Select the default-1 profile and rename it. For example, rename it to URL-Monitoring. Configure the action for all categories to alert. In the section that lists all URL categories, select all categories. To the right of the Action column heading, mouse over and select the down arrow and then select Set Selected Actions and choose Alert. Click OK to save the profile. After setting the Alert action, you can monitor user web activity through URL filtering reports on both appliances for a few days or weeks to determine accuracy of the provided category mappings. Palo Alto Networks recommends to validate accuracy for the top 1,000 websites seen by your organization. Apply the URL Filtering profile to the security policy rule(s) that allows web traffic of users. Select Policies > Security and select the appropriate security policy to modify it. Select the Actions tab and in the Profile Setting section, click the drop-down for URL filtering and select the new profile. Click OK to save. View the URL filtering logs to determine all of the website categories that your users are accessing. For information on viewing the logs and generating reports, see Monitor Web Activity. Select Monitor > Logs > URL Filtering. A log entry will be created for any website that exists in the URL filtering database in a category that is set to any action other than allow. In this procedure, all categories will be set to alert, which will cause traffic for all websites to be logged. This may potentially create a large amount of log files, so it is best to do this for initial monitoring purposes to determine the types of websites your users are accessing and compare URL categories triggered. Collect all URL category objects used in the Symantec Web Filter Policy Manager and map them into PAN-DB URL categories using the provided URL category map after vetting them for accuracy using the above steps. After determining the categories that your organization allows users to access, set the policy action to “allow” for these URL categories on the next-generation firewall. The firewall does not generate logs for traffic matching these URL categories. You can then make decisions on the URL categories that should be controlled according to the company policy by setting the appropriate policy action to each of these categories in the URL Filtering profile(s). The recommended actions column of the "URL category" table in the previous section and the "best practices" section at the end of this document are provided to further assist you in making policy decisions. If possible, it is recommended to use a “slow roll” approach, using USER-ID as described below when deploying these newly created URL Filtering profile(s) to Security policies. Clone an existing policy that allows web access and add an additional match criteria on User set to a single department. (i.e., IT, Marketing, Engineering, etc.) Add the new URL Filtering profile to this Security policy and move the policy above all policies that allow web access since policy rules are matched top down. Monitor the above policy for usage and get feedback from the users belonging to the Group Object. Incorporate changes as necessary to the URL Filtering profile before adding it to all other applicable security policies.     Best Practice URL Filtering Profile   Attach a URL Filtering profile to all rules that allow access to web-based applications to protect against URLs that have been observed hosting malware or exploitive content. As a best practice, use PAN-DB URL filtering to prevent access to web content that is at high-risk for being malicious. These include command-and-control, copyright-infringement, dynamic-dns, extremism, malware, phishing, proxy-avoidance-and-anonymizers, unknown, and parked. The best practice URL Filtering profile sets all known dangerous URL categories to block. Failure to block these dangerous categories puts you at risk for exploit infiltration, malware download, command and control activity, and data exfiltration. In addition to blocking known bad categories, you should also alert on all other categories so you have visibility into the sites your users are visiting. If you need to phase in a block policy, set categories to continue and create a custom response page to educate users on your acceptable use policies and alert them to the fact that they are visiting a site that may pose a threat. This will pave the way for you to outright block the categories after a monitoring period.   What if I can’t block all of the recommended categories?   If you find that users need access to sites in the blocked categories, consider creating an allow list for just the specific sites if you feel the risk is justified. On categories you decide to allow, make sure you set up credential phishing prevention to ensure that users aren’t submitting their corporate credentials to a site that may be hosting a phishing attack. Allowing traffic to a recommended block category poses the following risks:   Malware – Sites known to host malware or used for command and control (C2) traffic. May also exhibit Exploit Kits. Phishing – Known to host credential phishing pages or phishing for personal identification. Dynamic-dns – Hosts and domain names for systems with dynamically assigned IP addresses and which are oftentimes used to deliver malware payloads or C2 traffic. Also, dynamic DNS domains do not go through the same vetting process as domains that are registered by a reputable domain registration company and are, therefore, less trustworthy. Unknown – Sites that have not yet been identified by PAN-DB, perhaps because they were just registered. However,  these are sites that are oftentimes generated by domain generation algorithms and are later found to exhibit malicious behavior. Command-and-control – Command-and-control URLs and domains used by malware and/or compromised systems to surreptitiously communicate with an attacker's remote server to receive malicious commands or exfiltrate data. Copyright-infringement – Domains with illegal content, such as content that allows illegal download of software or other intellectual property. This category was introduced to enable adherence to child protection laws required in the education industry as well as laws in countries that require internet providers to prevent users from sharing copyrighted material through their service. Extremism – Websites promoting terrorism, racism, fascism or other extremist views discriminating people or groups of different ethnic backgrounds, religions, or other beliefs. This category was introduced to enable adherence to child protection laws required in the education industry. Proxy-avoidance-and-anonymizers – URLs and services often used to bypass content filtering products. Parked – Domains registered by individuals, oftentimes later found to be used for credential phishing. These domains may be similar to legitimate domains. For example, pal0alto0netw0rks.com, with the intent of phishing for credentials or personal identify information. Or, they may be domains that an individual purchases rights in hopes that it may be valuable someday, such as panw.net.   USE URL Categories to Define Traffic to Decrypt or Not Decrypt   Plan to decrypt as much traffic that is not private or sensitive as your firewall resources allow to reduce the attack surface by exposing and preventing encrypted threats. Understand local laws and regulations about the traffic you can legally decrypt and user notification requirements.   Please see documentation for SSL Decryption deployment and pre-requisites. The below steps describe Decryption policy definitions only.   Create a “no-decrypt” policy that will prevent any website mapping to the recommended "no-decrypt" URL category list (i.e., financial-services, health-and-medicine, government, etc.) from being decrypted. Even while not using decryption it is a recommended best practice to create a Decryption Profile to block sessions with expired certificates or untrusted issuers and use it with your "no-decrypt" policy. Navigate to Objects > Decryption Profile. Add a Profile called “nodecrypt” and check “Block sessions with expired certificates” and “Block sessions with untrusted issuers” under No Decryption tab. Navigate to Policies > Decryption and click Add. Enter a Name and optionally enter a Description and Tag(s). On the Source tab, enter the zone where the users are connected. On the Destination tab, enter the zone that is connected to the internet. On the URL Category tab, click Add and select the financial-services, government, and health-and-medicine URL categories. On the Options tab, set the action to No Decrypt. Also set Decryption Profile to a “nodecrypt”. Click OK to save the policy rule. Create a “must-decrypt” policy that will decrypt any website mapping to the recommended "must-decrypt" URL category list. (Malware, Phishing, Unknown, Command-and-control, Copyright-infringement, Proxy-avoidance-and-anonymizers, Content-deliver-networks, Parked, Web-based-email, Social Networking, Personal-sites-blogs, Web-hosting, Insufficient-content, Not-resolved, Online-storage and backup, Hacking, Questionable, Dynamic DNS) Navigate to Policies > Decryption and click Add. Enter a Name and optionally enter a Description and Tag(s). On the Source tab, enter the zone where the users are connected. On the Destination tab, enter the zone that is connected to the internet. On the Service/URL Category tab, enter all the Recommended URL Categories On the Options tab, set the Action to Decrypt and the Type to SSL Forward Proxy. Use a Decryption Profile along with your decryption policy to block sessions that fail on SSL decryption. Ensure that this "must-decrypt" policy is listed after the "no-decrypt" policy to ensure that rule processing occurs in the correct order. Create a “best-effort-decrypt” policy that will decrypt all other traffic using the same steps as above but with URL categories set to "any" and a Decryption Profile with options under Failure Checks section unchecked. This ensures that sessions are allowed even if SSL decryption fails.   With these three decrypt policies in place, any traffic destined for the financial-services or health-and-medicine or government URL categories will not be decrypted. All other traffic will be decrypted.     URL Filtering Use Cases   Case-1: Policy to block download of high-risk file types from certain categories [Decryption + URL Filtering + File-Blocking + Threat Prevention]   Please refer to SSL Decryption Best Practices to enable SSL decryption. This is necessary to accurately inspect, classify, and block encrypted traffic. Create a File Blocking Security Profile. The default basic file blocking profile can be used, or we can create a custom profile based on the basic file blocking profile. Select Objects > Security Profiles > File Blocking. Select “basic file blocking” or “strict file blocking” profile and click Clone. Rename the profile and edit to select appropriate file types for Block, Continue, and Alert actions based on your company policy or use the default profile provided. Click OK. Create the security policy rule that will block risky file downloads from specific categories. This rule must precede other rules because it is a specific rule. (More specific rules must precede other rules.) Select Policies > Security and click Add. Enter a Name and optionally a Description and Tag(s). On the Source tab add the zone where the users are connected. On the Destination tab, select the zone that is connected to the internet. On the Service/URL Category tab, Add the specific categories from which risky file download needs to be blocked. (i.e., Web-hosting, Personal-sites-blogs, Social Networking, Peer-to-Peer, Online-storage and backup, Web-based-email, Copyright-infringement, Shareware-and-freeware). On the Actions tab, select Action “Allow” and add the default profiles for Antivirus, Vulnerability Protection, Anti-Spyware, URL Filtering, and the newly created File Blocking profile. Click OK to save the security profile. Commit the Configuration. With this Security policy rule in place, any user trying to download executable files or other risky files from Dropbox, Box, or any free software download websites will be blocked. Because this rule will also allow access to the internet, threat prevention profiles are applied to the rule, so traffic that matches the policy will be scanned for threats. This is important because the allow rule is terminal and will not continue to check other rules if there is a traffic match.     Case-2: Policy to Control Web Access [Decryption + User-ID + App-ID + URL Filtering + Data Filtering + Threat Prevention] In this use case, users belonging to the Marketing group, for example, have access to Box for collaboration but not to any of the other “online-storage-and backup” vendors. All other users are blocked from all “online-storage-and-backup” applications. The company policy also states that documents marked “Confidential” should not be shared on Box by the Marketing group.     Please refer to the SSL Decryption Best Practices to enable SSL decryption. This is necessary to accurately inspect, classify, and block encrypted traffic. Create a Security Policy that will block all users from accessing “online-backup-and-storage” applications. This can be done either using a specific Security Policy or as part of a URL Filtering profile that would be included in all Security Policies that allow internet access. Select Policies > Security and click Add. Enter a Name and optionally a Description and Tag(s). On the Source tab add the zone where the users are connected. On the Destination tab, select the zone that is connected to the internet. On the Service/URL Category tab, click Add and add the online-storage-and-backup category. On the Actions tab, select Action Deny. Click OK to save the security profile. Create a Data Pattern Custom Object and add it to a data filtering Security Profile. Select Objects > Custom Objects > Data Patterns and click Add. Select Pattern Type as “Regular Expression”. Select File Type as “Any”. Set Data Pattern to “.*((Confidential)|(CONFIDENTIAL))”. Click OK. Select Objects > Security Profiles > Data Filtering and click Add. Set the Data Pattern Field to the above created object. Set Alert/Block Threshold to 1 and Log Severity to Critical and Click OK. Create the Security Policy that will allow the Marketing group to access Box application. Because this allow rule will also allow access to the internet, threat prevention profiles are applied to the rule, so traffic that matches the policy will be scanned for threats. This rule must precede other rules because it is more specific than the other policies. Select Policies > Security and click Add. Enter a Name and optionally a Description and Tag(s). On the Source tab add the zone where the users are connected. On the User tab in the Source User section click Add. Select the directory group that contains your marketing users. On the Destination tab, select the zone that is connected to the internet. On the Applications tab, click Add and add the boxnet App-ID signature. On the Actions tab, add the default profiles for Antivirus, Vulnerability Protection, and Anti-Spyware. Also add the Data Filtering profile that was created in the previous step. Click OK to save the Security Profile and commit the configuration. With these policies in place, any user who is part of the Marketing group will have full access to Box application and any user that is not part of the Marketing group will be blocked from all online-storage-and-backup websites. Additionally, all files that are shared on Box will be scanned for the keyword “Confidential” and blocked if found. An entry will also be logged under Monitor > Logs > Data Filtering.     Case-3: Subscribe to an external malicious URL feed [URL Filtering + External Dynamic Lists]   In this use case, administrator wants the firewall to ingest an external feed that provides IOCs (Indicators of Compromise) in the form of URLs. This dynamic list of URLs has to be continuously updated in policy and blocked by Palo Alto Networks next generation firewall without any manual intervention.   To protect your network from new sources of threat or malware, you can use External Dynamic List in URL Filtering profiles to block or allow or to define granular actions such as continue, alert, or override for URLs before you attach the profile to a Security Policy rule. Unlike the allow list, block list, or a custom URL category on the firewall, an external dynamic list gives you the ability to update the list without a configuration change or commit on the firewall.   Navigate to Objects > External Dynamic Lists. Click Add. Select Type “URL List”. Enter Source [this could be a web server hosting a file of URL’s]. Select appropriate Frequency of checks using the Repeat field. Click OK. Navigate to Objects > Security Profiles > URL Filtering. Select appropriate URL Filtering profile. The above created EDL should be seen as a custom category. Assign appropriate policy action to this category. This URL Filtering profile can now be added to a Security Policy(s).     With this Security Policy in place, any user attempting to connect to websites part of the URL feed will be blocked. This URL list is dynamically updated by the firewall without any commit required by the administrator. Any attempt to connect to these URLs is also logged under Monitor > Logs > URL Filtering.       Useful Resources PAN-DB URL Categorization Workflow Monitor Web Activity Configure URL Filtering Customize URL Filtering Response Pages Create Custom URL Categories Use an External Dynamic List in a URL Filtering Profile Safe Search Enforcement Prevent Credential Phishing Troubleshoot URL Filtering Incorrect Categorization SSL Decryption Overview