URL Filtering Database Lookup Flow

Details

The URL look-up flow occurs as follows:

• The dataplane checks it's own cache for a matching entry.
  • If entry exists, the category used will be the one that was cached (url_cache_hit counter will increment)
  • If the entry does not exist:
    • The url_cache_miss counter is incremented
    • The management plane's database is checked (url_db_request counter will increment)
    • If the management plane does not respond within 5 seconds
      • The url_request_timeout counter increments
      • The category will be identified as "not resolved"
    • If the management plane does not have the URL categorized in the PAN-DB/BrightCloud database, the category will be set to "unknown"

Here are the descriptions of the other counters related to the URL look-up process:

• url_cache_not_resolved: URL requests that where not resolved from the management plane. It is the number of "not resolved" url requests.
• url_cache_unknown_hit: URLs being categorized as "unknown". This counter reflects that brightcloud database doesn't have a record for the URL
• url_request_pkt_drop: Number of packets dropped while waiting for a response from management plane.

If the management plane CPU is running high, then this would explain the increase in dropped packets while waiting for the MP to respond, but this counter does not reflect if the actual categorization was not found.

A better counter to look at if worried about the management plane not responding and the URL being labeled as not categorized is url_request_timeout & url_cache_not_resolved

url_flow_state_invalid & url_flow_unmatched Those two are also indications of slowness. It means when the data plane gets the response from the management plane, the session is gone or session isn't the original session.

Enabling the setting "Dynamic URL Filtering" will access the PAN-DB/BrightCloud database in the cloud for the latest categorization information. Otherwise, as discussed above the URL filtering engine will only check for categorization on-device cache.

owner: apasupulati