I've read the articles about the processes that take place when analyzing traffic and understand that sometimes there could be an allow status when it seems there shouldn't be. However it also seems that if the traffic truly shouldn't be allowed there would be an associated log entry with some kind of denial.
In my case there is no associated denial and I'm would still like to know why this traffic seems to be allowed when apparently not matching my Custom URL Category.
Forgive me if I'm still just misunderstanding something about this.
Here's what I'm seeing in my logs:
What if you also create a URL Filtering Profile (call it Alert All URL), configured with "alert" for all categories, and attach it to this rule?
Because of the any I think the reason is one of the following:
If you share some more screenshots (logs with column 'rule', security policy in question, custom URL category), we should be able to help ;)
Thanks for the reply.
Unfortunately we don't have a URL Filtering license as we have another solution for that function.
For that what you intended to do (at least what I understood) - blocking or allowing specific websites with a custom URL category directly referenced in the security policy, you don't need an URL filtering license. Even for the logging part, mentionned by @jvalentine, there are workarounds that you will see the URL logs, without the license (only a workaround, not a recommended way)
Thanks @vsys_remo attaching more screenshots.
And to be clear, the Custom URL Category is showing up for most of the log entries, I'm just confused about why it's not showing up for some.
I just tried with an unlicensed lab firewall. I was able to create a URL filtering profile, configure it to alert on my custom categories, add the URL filtering profile to a security policy, and commit the changes. There is a commit warning complaining about no URL license, but it's a successful commit.
The colum that says "any" is only populated by attaching a URL filtering profile. I'm guessing this is why you're not seeing anything in that column.
Go ahead and create a URL filtering profile w/ "alert" as the action for all categories (or alert on your custom categories, and allow on all the rest), and attach it to your security policy. Commit and test.
I didn't realize that you could create a URL Filtering profile without a license. Not sure I would like getting that warning (I assume I would continue to get this on subsequent commits).
Again, just to be clear, I do get my URL Category showing up in the logs (I just filtered those out in my previous images). Here's another look at the logs without the filtering and an example of "any" circled.
Also, my main goal here is to be able to confidentally tell upper management that no traffic is being allowed to a destination that is not in my URL Category.
Understanding your main goal is helpful. Looking at your security policy, I don't see all of the applications you're allowing. Since SSL is in the list, I'm assuming web-browsing is in the list too. If it is, then this may be happening:
Client PC sends SYN on TCP/80, firewall doesn't know what the application is yet, but since you have a rule that allows web-browsing on TCP/80, that packet is permitted to pass. At this point the firewall doesn't know what the application is, let alone the URL.
Server responds with SYN/ACK. At this point, firewall still doesn't know what the application is yet, but since there are rules that allow web-browsing on TCP/80, firewall will continue to permit the handshake and subsequent data packets until it can definitively match your policies.
Client responds with ACK. Firewall still doesn't know application or URL.
Client sends some request on TCP/80. This is the firewall's first opportunity to look at layer-7 data and start determining things like application and URL. Some applications are immediately recognizeable and the firewall can take action beginning with this packet. Other applications require a couple of packets in order to definitively identify the application.
What is likely happening is that the firewall allows the TCP/80 traffic, even identifies it as web-browsing, and then it attempts to match that traffic with your permit rules. If it matches, great. If not, it stops. The trick question is, how should the firewall log the traffic? Should it log the traffic as being denied (when some portion went through?)
Can you add the packets sent / packets received columns to your log view? That would be an interesting datapoint for the traffic that you've circled. Also add the "Session ID" column. From the CLI run a "show session id xxxxxxxx" for the traffic log and see what additional detail is available.
It would also be helpful to know the exact URL that generated this traffic log. I seem to remember a way to do this. It involves creating a URL Filtering Profile...
EDIT: I'd recommend reading up on application dependencies, as this may be part of the issue. Also, this (https://live.paloaltonetworks.com/t5/Featured-Articles/DotW-Blocked-traffic-has-an-allow-log/ta-p/72...) is another possibility.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!
The Live Community thanks you for your participation!