This is a full walk-through on how to use Expedition to run the new functionality to learn from the device logs and generate security policies based on consumption models.
Duration of this video is 6 min. aprox.
Will this be spelled out in the hopefully forthcoming User Guide for Expedition? Also, will this be something I can just pull out of the Palo device direct? It seems unnecessary to have to go to a syslog server when the Palo device has them essentially in the device or in Panorama. Maybe I missed that in the video. You moved pretty quick which I appreciate.
There's no link to the video anymore.
In further User Guides we will describe in deeper detail the process of Learning from Logs.
The video shows how to export the logs via SCP (not syslog server), preprocess the logs to convert them into an internal format (parquet) enhanced for paralell processing and machine learning, and crunching this parquet for identifying traffic behaviors and suggest security policies.
Notice that I mentioned the parquet format. This is the reason we require exporting the logs into Expedition, as we need to convert the original log format into a parquet format that will enable us for the ML processes. So, we can't directly work with internal DB in a PANOS device. Additionally, we don't want to stress the PANOS devices with this intense data analytics process, but we can stress a VM hosting Expedition.
The video was originally intended for a presentation at Ignite, therefore it is condensed to show a rapid view of the process in only 6 minutes. And the video seems to be available now.
I'm trying to test Greenfield ML in order to have a flow matrix of my Firewalls and have an idea about security policies, but the phase called "Spark: Process CSV files to Parquet" takes too long (more than 6 hours), is that normal ? or should i stop it and repeat again ?
The status shown is "Pending", so i don't know if it's already started or not ! samething as in security policies, when i try to analyze data from specific security policy (CONTENT LEARNED FROM expedition ML), the status also shown is "pending" for too long without any reaction ! can you please help to resolve that issue ?
Thanks a lot,
If the status is "Pending" most probably the process did not start.
I guess you do have a version of Expedition prior to 1.0.99.
I would suggest to update Expedition via the apt-get commands and try again.
I will provide you better information if something is not correctly set up.
I just prepired all my collected CSV-Files. and tryed to do an analysis, but I cant add an logcollector, because of not reachable PA.
Is there a possibility do do the analysis offline?
thanks in advance