topic Re: Total Application Time in General Topics

Total Application Time

mario11584 — Wed, 27 Sep 2017 18:09:58 GMT

I'm trying to figure out the total application time of some specific applications. For example, for the last 7 days I'd like to know for a particular subnet how much time was spent on YouTube. Is this possible? So I'm looking for something to tell me that there has been a total of 8 hours, for example, of YouTube sessions for the last 7 days. I can find session count, I can find bytes, but I can't find anything that takes all the session duration data the PA has and give it back to me in this way.

Re: Total Application Time

BPry — Wed, 27 Sep 2017 18:30:24 GMT

@mario11584,

That is currently not directly supported by Palo Alto. I would recommend adding your vote to the requisite future request via your SE, I'm sure there is already one out there for this.

Re: Total Application Time

Raido_Rattameister — Wed, 27 Sep 2017 18:37:23 GMT

Elapsed time might help you out.

Re: Total Application Time

jvalentine — Wed, 27 Sep 2017 18:39:48 GMT

Here's a couple of ways to try and figure this out:

1.) Custom Reporting. You can create a custom report from the traffic log where the (app eq youtube) and the (addr.src in 10.1.2.0/24) and include "Elapsed Time" in the selected columns:

That would give you a report that looks like this:

Keep in mind that this value includes more than just the actual "stream" elapsed time. This is elapsed time for all TCP sessions where the application was Youtube. Some of those sessions could be static pages, ads, pre-loading the next video that wasn't watched, etc.

2.) User Activity Reporting: This doesn't necessarily work by subnet, though. If you could put all of the users of the subnet in question into a single LDAP group, then you could do a group activity report - and there's an estimated 'browse time' column for the URL's visited by that group.

3.) Rough Math: Figure out what the average MB/minute is for Youtube, then run a traffic report determining total Youtube traffic for that Subnet. Divide that by the MB/minute and you get total minutes of Youtube.

Reporting pro-tip: No matter which way you go, I'd highly recommend using yourself as a guinea pig. Watch youtube videos for 15 minutes and then run each of these reports against yourself to determine what kind of "fudge-factor" you'll need to include with the results.

Re: Total Application Time

gwesson — Wed, 27 Sep 2017 18:57:02 GMT

This is one of those things that everyone wants but no one can truly deliver without a client monitor (and even then it's not often accurate).

Here are some examples of how reporting on actual browse time can be a challenge:

User has a playlist loaded on a tab but isn't actually watching videos actively (maybe a music playlist)
User is browsing a web forum with embedded YouTube videos that autoplay.
User is watching YouTube on their phone while working (assuming phone is on the same subnet as their computer)

If you are only concerned about bandwidth, you can get good reports from the firewall for that. But translating YouTube session duration with actual time spent viewing videos isn't something that translates well with just traffic log analysis.

Re: Total Application Time

mario11584 — Wed, 27 Sep 2017 21:11:01 GMT

This is great! Though I do run into accuracy issues, as has been mentioned, after running some tests. A lot of my traffic is encrypted too, so it shows up as SSL traffic and not YouTube. Though I did some see some application traffic for YouTube over 443, which I find interesting. Why does some of it show up as SSL and some as YouTube, both over 443? I would obviously need SSL decryption to dig deeper into the SSL traffic.

Re: Total Application Time

LucaMarchiori — Wed, 27 Sep 2017 21:19:42 GMT

Chance are some of the traffic will also be tagged as "quic" - also on port 443.

Re: Total Application Time

jvalentine — Wed, 27 Sep 2017 22:52:06 GMT

@mario11584 wrote:
This is great! Though I do run into accuracy issues, as has been mentioned, after running some tests. A lot of my traffic is encrypted too, so it shows up as SSL traffic and not YouTube. Though I did some see some application traffic for YouTube over 443, which I find interesting. Why does some of it show up as SSL and some as YouTube, both over 443? I would obviously need SSL decryption to dig deeper into the SSL traffic.

Enabling SSL decryption would make the report more accurate, but one does not "just" enable SSL decryption without testing first.

Your other option is to get a better handle on how the firewall sees Youtube traffic as a whole (app-id=youtube, app-id=ssl+some other indicator, etc.) Does the actual video stream get tagged as "youtube" or "ssl"? If it is identified as youtube, that makes it easy for your reporting goals. If the stream is identified as ssl (or a mix of the two), then you'll need to dig deeper into your logs to figure out what's going on. (I recommend using the unified log viewer and adding both the URL and Session ID colums to the list).

It could be that the Youtube app-id needs some updating/additional coverage - in which case open a support ticket. It could be that only decryption will resolve this issue. Or finally, you could find some additional information in the unified logs that allows you to generate a report combining all youtube and specific ssl traffic together.