BrightCloud URL Filtering Exhibiting Multiple Unknown URLs

Issue

BrightCloud URL filtering is exhibiting multiple unknown URLs, even after clearing DP cache.

For example, the user runs the following CLI command to display URLs with "unknown" category:

> show running top-urls category unknown

Hits  Categories  URLs

--------------------------------------------------------------------------------

3434  unknown     g.ceipmsn.com/

570   unknown     www.africadownunderconference.com/

449   unknown     resources.newscdn.com.au/

265   unknown     tags.tiqcdn.com/

184   unknown     helpdesk.vizstone.com/

144   unknown     cdn.newsapi.com.au/

118   unknown     osd.oxygem.it/

93    unknown     www.oppomobile.com.au/

89    unknown     myaccount.zetta.net.au/

82    unknown     www.tintacar.com.au/

Note: The URL log also shows unknown categories because it records the URL category from the dataplane.

The management plane test url command is able to resolve the url:

> test url tags.tiqcdn.com

tags.tiqcdn.com content-delivery-networks (Cloud db)

The dataplane test url-resolve-path is unknown

> debug dataplane test url-resolve-path tags.tiqcdn. com/

URL tags.tiqcdn.com/, category unknown

Clearing the dataplane cache and management plane dynamic cache with the following commands does not resolve the issue. The "unknown URL" hit continue to increment after clearing both cache:

> clear url-cache all

> delete dynamic-url host all

Cause

BrightCloud URL DB behavior for some websites is as follows:

• If the global setting for dynamic-url (set deviceconfig setting url dynamic-url yes) is enabled and url-profile is not, the site returns unknown.
• If the url-profile has dynamic-url enabled, but the global setting is disabled, the site returns not-resolved.
• If both the global and url-profile has dynamic-url enabled, then the site is returned matching the cloud category.

Resolution

If the url-profile has dynamic-url enabled, make sure that the global setting is also enabled.

1. Go to Objects > Security Profiles > URL Filtering and choose URL Profile > Check Dynamic URL Filtering.
2. Using BrightCloud, set Dynamic MP URL globally using the following commands:

> configure

# set deviceconfig setting url dynamic-url yes

# commit

For more information, refer to https://live.paloaltonetworks.com/docs/DOC-3685

owner: jlunario