on 05-28-2018 04:58 AM - edited on 07-08-2019 08:34 AM by Retired Member
This is a full walk-through on how to use Expedition to run the new functionality to learn from the device logs and generate security policies based on consumption models.
Duration of this video is approximately 6 minutes.
Will this be spelled out in the hopefully forthcoming User Guide for Expedition? Also, will this be something I can just pull out of the Palo device direct? It seems unnecessary to have to go to a syslog server when the Palo device has them essentially in the device or in Panorama. Maybe I missed that in the video. You moved pretty quick which I appreciate.
In further User Guides we will describe in deeper detail the process of Learning from Logs.
The video shows how to export the logs via SCP (not syslog server), preprocess the logs to convert them into an internal format (parquet) enhanced for paralell processing and machine learning, and crunching this parquet for identifying traffic behaviors and suggest security policies.
Notice that I mentioned the parquet format. This is the reason we require exporting the logs into Expedition, as we need to convert the original log format into a parquet format that will enable us for the ML processes. So, we can't directly work with internal DB in a PANOS device. Additionally, we don't want to stress the PANOS devices with this intense data analytics process, but we can stress a VM hosting Expedition.
The video was originally intended for a presentation at Ignite, therefore it is condensed to show a rapid view of the process in only 6 minutes. And the video seems to be available now.
Hi Team,
I'm trying to test Greenfield ML in order to have a flow matrix of my Firewalls and have an idea about security policies, but the phase called "Spark: Process CSV files to Parquet" takes too long (more than 6 hours), is that normal ? or should i stop it and repeat again ?
The status shown is "Pending", so i don't know if it's already started or not ! samething as in security policies, when i try to analyze data from specific security policy (CONTENT LEARNED FROM expedition ML), the status also shown is "pending" for too long without any reaction ! can you please help to resolve that issue ?
Thanks a lot,
If the status is "Pending" most probably the process did not start.
I guess you do have a version of Expedition prior to 1.0.99.
I would suggest to update Expedition via the apt-get commands and try again.
I will provide you better information if something is not correctly set up.
The video is embedded in the post.
Either check with another browser or wait a bit for the video to load.
Hello,
i need to know how we know which rules from transformed rules (in vsys2) refered to which original wide open rules (in vsys1)?
if the tool ML analysis three rules as example, how we determine which rules from the output rules refered to which one of the three orignal rules?
Ahmed sabry.
Unless you are fine doing Rule Enrichment (RE), you would have to do Ml having only one rule selected at a time.
Do you have a clear idea regarding the goal of ML and RE?
because maybe you actually want to do RE and then you have no problems selecting the 200 and doing one single apps. The result of RE will tell you which rule got enriched.
