Looking for the better idea.
User reported xyz.com site is not working, When I look at the firewall logs xyz.com was allowed in firewall.
And I filtered the logs with source IP for 5mins time frame and found deny logs for particular IP address 184.108.40.206
I am not sure what is the url for 220.127.116.11, In such case what you will do to find the URL for 18.104.22.168
I usually do a search in splunk log server, for this IP 22.214.171.124 it will show all the related logs, I filter with dns logs then I can find the hostname for 126.96.36.199 ex. ab.xyz.com, Now I am able to permit this additional URL, it will fix the issue.
What if I don't have splunk log server, how can I identify this additional link(ab.xyz.com)
Also is there a way to find all the links that is being used in a website?
thanks for the post in LIVEcommunity!
Right of the hand I can think off using Unified log under Monitor > Logs > Unified. Unified logs is an aggregation of all logs (Traffic, Threat, URL,..). This should provide a single view of blocked traffic regardless its log origin.
Hi @JANARTHANAN1392 ,
Are you using 188.8.131.52 just example of this is the actual address you see the logs? I am asking because 184.108.40.206 is Cloudflare public DNS, so I would expect to see deny traffic, if you don't allow any DNS, but use only internal/trusted DNS
Now on the question - URL log is created only by URL filtering profile if blocked or allowed. If traffic is blocked by security rule, traffic log will be generated. But if you remember from documents - deny security rule does not apply any profile, so traffic is denied before being inspected by url filter to create any url log.
So it is Catch 22. The sad think is that if you have custom URL category as match criteria in security rule, you still don't have a way to know which url was blocked and which was allowed.
I recently found myself in similar situation - I had to found all the URL that were required and allow only them, blocking everything else. But user was not able to provide exact list. I would suggest you two options:
- Create temp rule for your test user allowing any traffic (you probably want to filter http and https ports at minimum). Apply URL filtering profile that has "alert" action for all categories. Ask the user to test and open the problematic page. Check the URL filtering logs take a note for all of the URLs that FW has detected.
- At the very bottom of your policy, right above the "cleanup" rule create new rule that allow any destination for ssl and web-browsing. But this time apply URL filtering profile which set all categories to block. Essentially this rule will still act as "deny everything that is not explicitly blocked", but since the URL filtering will block the traffic and not the security rule, you will have URL log (instead of traffic log), which will give you information for all URL that FW has blocked, because they were not explicitly allowed.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!