Symptoms
Testing a URL using the test url command shows the correct category, but when a user visits the same web page, the page gets categorized as unknown in the logs. The issue persists even with the dynamic URL filtering option turned on in the URL filtering profile.
Issue
This could be a result of not configuring URL filtering profile on all the security rules.
Example:
- Rule 1: User A is allowed anywhere on the internet and no URL filtering policy is configured on this rule
- Rule 2: A URL Filtering policy is in place to 'deny access to'
When User A browses websites, Rule 1 matches all the connections and because no URL filtering profile is enabled on that rule, logs do not contain a URL category for the websites that were accessed. Traffic logs show source and destination IP address, but URL filtering logs don't show those connections.
Resolution
Configure a URL filtering profile on all necessary security policies where URL filtering is desired. A URL filtering profile can be created that allows everything, and applied to the rule allowing all categories (Rule 1 in the example below). Rule 2 would use the more restricted URL filtering policy.
Alternatively, you can configure a setting via the CLI that uses dynamic-url global setting for rules that don't have URL filtering profiles enabled:
> configure
# set deviceconfig setting url dynamic-url yes
# commit
This configuration option is available beginning with PAN-OS 4.1.3. You must also clear the URL cache for the new configuration to take effect going forward. The command to clear the URL cache is:
> clear url-cache all
If clearing the URL cache doesn't help, then the dynamic url must be deleted manually (per host or all) with the following command:
> delete dynamic-url host name/all
Note: URL filtering logs are generated only when the action is set to Alert, Block, or Continue.
owner: sdurga