URL Blank in Traffic Logs

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.
Palo Alto Networks Approved
Palo Alto Networks Approved
Community Expert Verified
Community Expert Verified

URL Blank in Traffic Logs

L1 Bithead

The traffic logs for our PAs almost never actually show a URL, despite the URL category getting properly assigned. The only time I ever see a URL show up in the logs is if it is specifically denied because of the URL category, which is fairly rare. If they are allowed, or blocked based on something else like application, no URL shows. Is this expected behavior or could it be a configuration issue on our end? I am a bit confused by this as adding a URL to a custom URL category is successful in allowing specific sites, so the PA is obviously aware of the URL, but it doesn't show in the logs. A common scenario is a site being blocked based on application but we need to make an exception for a specific URL. I can't grab the URL from the logs and will instead need to contact the user to gather that information. I will then add that URL to a custom URL category, put that in a rule with the relevant application and the URL will be allowed. Throughout that whole process, the logs will not show the URL in the traffic logs despite the PA allowing based on the URL category. All of the relevant policies have a URL Filtering profile applied and log at the end of the session. 

1 accepted solution

Accepted Solutions

Cyber Elite
Cyber Elite

Hi @ALSdburns ,

 

The default action of allow for URLs does NOT log the URL.  If you want to log the URL, set the action to alert.  https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/url-filtering/url-categories/url-filtering...

 

Thanks,

 

Tom

Help the community: Like helpful comments and mark solutions.

View solution in original post

5 REPLIES 5

Cyber Elite
Cyber Elite

Hi @ALSdburns ,

 

The default action of allow for URLs does NOT log the URL.  If you want to log the URL, set the action to alert.  https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/url-filtering/url-categories/url-filtering...

 

Thanks,

 

Tom

Help the community: Like helpful comments and mark solutions.

Thank you for the info! Exactly what I was looking for.


Sorry to bring this up again, but after testing the alert setting I had one follow up question. How would we get the URL to show for traffic that was blocked based on other factors, such as application? 

I imagine I should just apply our default URL filtering to our Block All rule, but want to doublecheck. I assume even if you "allow" certain categories in the applied URL Filter, it will still get blocked if the actual rule is a Deny rule?

Cyber Elite
Cyber Elite

Hi @ALSdburns ,

 

That is a great question.  I cannot find any documentation for that scenario.  I can understand if traffic is being blocked that you may want to see the URL in order to determine if it should be allowed.

 

I can only assume that once the NGFW determines that the traffic will be dropped, it stops the session setup and never gets to the URL lookup.

 

One idea is to resolve the destination IP (manually or by Resolve hostname on the bottom of the Monitor tab) in order to find the parent domain.

 

Thanks,

 

Tom

Help the community: Like helpful comments and mark solutions.

L6 Presenter

In addition to what @TomYoung says, a single session may also contain many, many URLs that were allowed prior to the session being blocked for an unrelated reason (App-ID, etc.). When a device opens a connection it can request a URL and receive a response. Before or after that response is received, the client can make another request over the same session. This is called HTTP pipelining, where multiple resources can be inflight at the same time. If there are multiple URLs which are allowed, which URL would you log? In other cases it may not even be HTTP communication, so there is no URL associated with the Application ID.

 

A second way to possibly resolve a hostname for the blocked site is to look at your PA DNS cache (if you are resolving/filtering internal DNS off your PA). From the CLI, "show dns-proxy cache all | match <ip address>" will show you any matching FQDNs or the IP.

  • 1 accepted solution
  • 2491 Views
  • 5 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!