03-15-2020 12:10 PM - edited 03-15-2020 12:13 PM
I'm trying to configure M. Learning in Expedition so that we can analyze the traffic passing through specific any any rules.
We use Panorama to manage the security policy on each of the individual firewalls. Is this an issue when trying to use Machine Learning? Here is the issue I'm running into.
I've setup Scheduled Log Exports on each of the individual firewalls to export their logs to Expedition.
In Expedition if I add the individual firewall in the Devices tab I can see the csv file and process it in its M.Learning tab. But if I create a project with that firewall I cannot see any of the policy to do a traffic analysis or much of anything else. My assumption is because all that information is being managed at the Panorama level.
However. If I add the panorama in the Devices tab and retrieve the specific firewall device and retrieve its configuration, I cannot then process the exported log in the M. Learning tab. I see the files but everything is grayed out and it has a header that says "Process CSV logs can only be executed from FW devices." But when I add the Panorama to a project I can see the entire security policy and everything else.
Does ML only work in environments where the firewalls aren't centrally managed by Panorama?
03-19-2020 02:01 AM
If you desire to use ML or RE, we do need to have logs in Expedition.
ML and RE do a data analytics process on the traffic logs that can't be performed on Panorama or FWs, as they are much more complex than generating reports on the devices.
Therefore, we need to export the traffic logs into Expedition so we can perform the analysis.
The steps that Lynn showed are required to execute ML or RE. Also, it is very important that the configuration you bring into your project comes from a Device you have declared in Expedition, and not by directly providing an XML to the project. This is necessary because this way we can know the relationship between the security rules you want to analyze (that are declared in a device config) and the traffic logs that the device has provided.
Notice that the "device" is the link between traffic logs and security rules, therefore we need to import the configuration from the Device
03-19-2020 07:39 AM
The problem appears to lie in the dynamic between using individual firewalls & panorama.
You can only do HALF of the workflows depending on if you import individual firewalls vs panorama.
You can ONLY do the first half of the workflow if you import individual firewalls.
You can ONLY do the second half of the workflow if you import panorama.
But you cannot do it ALL using either method. Meaning you would have to break everything down daily as you get new logs.
In the device tab. If you import just the individual firewalls...
You can do the following:
You can NOT do the following:
Now if you import Panorama In the device tab
Importing Panorama *absorbs* those individual firewalls. As you can see, the individual firewalls are gone and only panorama remains in the device tab.
Now things are reversed.
You can NOT do the following:
You can do the following:
This obviously is not a workflow that is practical. If I were to need to do traffic analysis daily I would basically have to break everything down and re-implement everything again. I'd have to do the following daily.
Now unless there is still something wrong with the way I'm implementing this I think there is a bug. The problem really lies in the disabling of the ability to process CSV Logs to individual firewalls when you are using Panorama.
If "Process CSV logs can only be executed from FW devices." and the grayed out CSV logs were to be removed when using Panorama it should solve everything.
03-19-2020 08:05 AM
From the Panorama device, you can provide a Path for all the firewalls managed by that Panorama.
If you select to see all the Firewalls, not only the directly connected devices (do it clicking on the three lines icon on the Devices view) you would see all the Firewalls, and you can even select and Autooprocess on the logs, so you do not need to get into them daily.
Later on, in your project, you can import the Panorama config and define in your log connector the Device Group and the devices within that DG for the ML and RE.
03-19-2020 08:25 AM
Can you provide a screenshot of what you are referring to? I'm not following.
03-19-2020 08:54 AM - edited 03-19-2020 09:06 AM
Instead of add firewall directly to devices, can you only add Panorama as devices, and go to Panorama Device , Click on orange button "Retrieve Connected Devices "
When you want to process logs , you will then click on the icon on the right upper corner to "show all devices" , and you should see your firewalls , then goin to the firewall to process the ML logs.
So you do not need to add firewall directly to the devices, just add panorama and retrieved the connected devices, then process the logs in the connected firewall that you got those logs from, make sure the traffic log matched the serial# of the firewall.
After ML log is processed then you can add a new project and continue the steps I mentioned in the previous post.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!