Web Browsing and Associated Traffic Logs

[Brandon_Wertz]

I've never really ran into this issue before and was hoping to get some tips on how you all correlate this type of information.

So we have FQDN host blocks for certain Internet based resources.  More and more I'm seeing collateral impacts to unintended websites.  (Yes I'm fully aware of how services can be tied together either through associated cloud services providers or even associated name resoulution services.)

My question is how is it that you guys can say from the point of "click" on a link the say "10 entries" in a traffic log are associated with that 1 "click."

I know that 7.1 has the Unified log function, but I'm still looking for that complete picture.

Any help would be appreciated.

[Raido_Rattameister]

Try to play around with "referer" field under URL filtering log to identify what site tried to load blocked content.

[Brandon_Wertz]

So here's the URL log view:

Unfortunately no where does this log say anything about an IP address or the FQDN block totally not associated with MSN that prevented this page from even loading.

[Raido_Rattameister]

Try:

(referer contains "msn.com") and (action neq alert)

[Brandon_Wertz]

No dice on that search query.

[Raido_Rattameister]

But what happens?

Users see block page?

Try to get timeframe when user had issues and find blocked session to identify what was exactly blocked.

[Brandon_Wertz]

*EDIT - Fixed the summary of the last two captures, they needed to be flipped*

So here's a PCAP from the FW the Rx stage and Drop stage from me going to the site.

Looking at stream 1 that's me trying to go to the site.  With the timehack of 15:31:40.123961

This is the drop stage where you can see continual request to the IP occuring at timehack 15:31:40.155869 which again is immediately after the inital web request.

This is looking at the Rx stage where you can see where the communication to the IP in question for the FQDN drops.  This is occuring on stream 20 at 15:31:40.155287, so after the "start" of the web request.

So does anyone have a good way to track in some sort of consolidated log without performing a report on a specific user every time this comes up?

[Brandon_Wertz]

So this is stemming from a security policy that I've since had changed, but in essence this was a FQDN drop, so a L3 drop.  The original intent was users wouldn't be getting a response page at all.  The security team didn't want "callbacks" to malicious domains so there was just a L3 drop rule for specific stuff.

I've since gotten this changed where these types of things are getting integatred into a L7 block so users would see a response page.

In short I'm trying to troubleshoot something that I don't foresee being a big issue moving forward, but I'm still curious to understand how understand the academic question of how can I see an "end-to-end" log of a single session so to speak from when a user tries to vist a single page without creating a specific user report.

[Raido_Rattameister]

Yeah you are blocking already at TCP SYN level and it never gets to HTTP GET.

You can check what you see in traffic log if you filter traffic that goes to 64.33.232.56

Click on the mag glass and check if you see any more details there.

[Brandon_Wertz]

@Raido_Rattameister I get what you're saying but I'm looking for a straight forward way for an admin to "follow the trail" of a user when they're intiating this traffic.

The only reason I know that this dropped traffic is associated to the web page loading is because I can create policy in the firewall and put my traffic above these FQDN denies.  When doing so this destination traffic isn't dropped and the web pages load.

Session IDs (Funny enough there isn't even a session ID for the dropped traffic.  To me this seems like where this should be a feature enhancement or something where an admin can correlate a drop to something a user is actually doing.) are different between the URL logs and the traffic logs of the traffic that's being dropped so there's really no way for me to correlate in any fashion, that I can find, a these various logs into something useable which define a single action from a user.

Am I missing something?

[Raido_Rattameister]

What is the reason this traffic is blocked?

Do you allow traffic to only limited number of ip addresses through FQDN objects in policy destination field?

If you would block traffic based on URL category then session would be created and you could follow referer field in URL filter log.

If you throw away traffic based on ip/port then session is not created and that is the reason you don't see session number.

[Brandon_Wertz]

Traffic is blocked because of past spearfishing.

0 hosts are allowed to the FQDN object.

Yes

Item 4 really doesn't provide for any conext within the enviornment for what precipiatetd the original traffic.

I've still yet to understand how to track traffic back to specific user actions