Custom URL Category in security rule - traffic log shows allowed with "any" in URL Category field

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Custom URL Category in security rule - traffic log shows allowed with "any" in URL Category field

L2 Linker

I've read the articles about the processes that take place when analyzing traffic and understand that sometimes there could be an allow status when it seems there shouldn't be.  However it also seems that if the traffic truly shouldn't be allowed there would be an associated log entry with some kind of denial.
In my case there is no associated denial and I'm would still like to know why this traffic seems to be allowed when apparently not matching my Custom URL Category.
Forgive me if I'm still just misunderstanding something about this.

Thanks.

Here's what I'm seeing in my logs:

UrlCat.jpg

12 REPLIES 12

L7 Applicator

What if you also create a URL Filtering Profile (call it Alert All URL), configured with "alert" for all categories, and attach it to this rule?

Hi @herrmoss

 

Because of the any I think the reason is one of the following:

  • You did not attach the custom URL category to the policy
  • Traffic hits a rule before your rule with the attached custom URL category

 

If you share some more screenshots (logs with column 'rule', security policy in question, custom URL category), we should be able to help 😉

 

Regards,

Remo

Thanks for the reply.

Unfortunately we don't have a URL Filtering license as we have another solution for that function.

 

For that what you intended to do (at least what I understood) - blocking or allowing specific websites with a custom URL category directly referenced in the security policy, you don't need an URL filtering license. Even for the logging part, mentionned by @jvalentine, there are workarounds that you will see the URL logs, without the license (only a workaround, not a recommended way)

 

You are correct @Remo I only mentioned the licensing issue becuase @jvalentine mentioned creating a URL Filtering Profile and I believe I can't do that without a license.

Thanks @Remo attaching more screenshots.

And to be clear, the Custom URL Category is showing up for most of the log entries, I'm just confused about why it's not showing up for some.

 

Logs.jpgrule.jpgURL_Cat.jpg

 

I just tried with an unlicensed lab firewall.  I was able to create a URL filtering profile, configure it to alert on my custom categories, add the URL filtering profile to a security policy, and commit the changes.  There is a commit warning complaining about no URL license, but it's a successful commit.  

 

The colum that says "any" is only populated by attaching a URL filtering profile.  I'm guessing this is why you're not seeing anything in that column.  

 

Go ahead and create a URL filtering profile w/ "alert" as the action for all categories (or alert on your custom categories, and allow on all the rest), and attach it to your security policy.  Commit and test.  

I didn't realize that you could create a URL Filtering profile without a license.  Not sure I would like getting that warning (I assume I would continue to get this on subsequent commits).

Again, just to be clear, I do get my URL Category showing up in the logs (I just filtered those out in my previous images).  Here's another look at the logs without the filtering and an example of "any" circled.

 

Also, my main goal here is to be able to confidentally tell upper management that no traffic is being allowed to a destination that is not in my URL Category.

 

LogNoFilter.jpg

Understanding your main goal is helpful.  Looking at your security policy, I don't see all of the applications you're allowing.  Since SSL is in the list, I'm assuming web-browsing is in the list too.  If it is, then this may be happening:

 

Client PC sends SYN on TCP/80, firewall doesn't know what the application is yet, but since you have a rule that allows web-browsing on TCP/80, that packet is permitted to pass.  At this point the firewall doesn't know what the application is, let alone the URL.  

 

Server responds with SYN/ACK.  At this point, firewall still doesn't know what the application is yet, but since there are rules that allow web-browsing on TCP/80, firewall will continue to permit the handshake and subsequent data packets until it can definitively match your policies.  

 

Client responds with ACK.  Firewall still doesn't know application or URL.  

 

Client sends some request on TCP/80.  This is the firewall's first opportunity to look at layer-7 data and start determining things like application and URL.  Some applications are immediately recognizeable and the firewall can take action beginning with this packet.  Other applications require a couple of packets in order to definitively identify the application.  

 

What is likely happening is that the firewall allows the TCP/80 traffic, even identifies it as web-browsing, and then it attempts to match that traffic with your permit rules.  If it matches, great.  If not, it stops.  The trick question is, how should the firewall log the traffic?  Should it log the traffic as being denied (when some portion went through?)

 

Can you add the packets sent / packets received columns to your log view?  That would be an interesting datapoint for the traffic that you've circled.  Also add the "Session ID" column.  From the CLI run a "show session id xxxxxxxx" for the traffic log and see what additional detail is available.  

 

It would also be helpful to know the exact URL that generated this traffic log.  I seem to remember a way to do this.  It involves creating a URL Filtering Profile...

 

EDIT:  I'd recommend reading up on application dependencies, as this may be part of the issue.  Also, this (https://live.paloaltonetworks.com/t5/Featured-Articles/DotW-Blocked-traffic-has-an-allow-log/ta-p/72...) is another possibility.

@jvalentine

It is possible to configure the url profile without a license and apply it to a policy ... but did yoz really get url logs?

Sorry for the delay in responding but I've been tied up with other things.
Also I will not be able to work on this for the next week.
When I can get back to this I will look at some of the things you've mentioned.

However in regard to your comment:
"What is likely happening is that the firewall allows the TCP/80 traffic, even identifies it as web-browsing, and then it attempts to match that traffic with your permit rules.  If it matches, great.  If not, it stops.  The trick question is, how should the firewall log the traffic?  Should it log the traffic as being denied (when some portion went through?)"

I would think that if the firewall is going to log this traffic as allowed because it got part way through the process, it should also have a denied entry when it determines that something about the rule (in my case the URL Category) prevents it.  It seems like this isn't happening and maybe that's just the way it works, however I find this confusing.

Thanks very much for your insight.

So @Remo and @jvalentine, in betwen my other priorities I have been researching this issue with my limited knowledge and understanding.

I have discovered that in 60% of the cases the destination IP Address allowed that shows "any" for URL Category in the logs has been also been allowed sometimes with the correct URL Category showing.  I'm not sure why this is and am at a loss to explain the remaining 40% that never show the correct URL Category.

At this point I am willing to accept this behavior but will continue to monitor.

Thanks for your time on this.

 

 

  • 4921 Views
  • 12 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!