We are following Expedition-LogAnalysisGuide_v1.0.2.pdf to start using ML
Expedition is v1.1.12, BP v3.6.3, Task Manager is running and there are no internal checks to remediate
The firewalls are VMs running 9.0.1
In page 6 in the above doc, IP address 192.168.44.131 is used for Expedition. However, in page 7, a different IP is used in section Hostname: 10.30.11.50
Please confirm if the IP to configure in Schedule Log Report-Hostname should be the IP for Expedition.
We worked under this assumption, but we get the following error when clicking on Test SCP server connection: Error response from server: bash: /home/expedition/logs/ssh-export-test.txt: No such file or directory
What is causing this error and how can it be fixed?
Until the above is sorted, we would like to upload the logs manually, but the pdf does not include instructions on how to accomplish this:
MANUALLY EXPORT LOGS FROM MONITOR
You can always go to any firewall from Palo Alto Networks and from the Monitor tab export the logs in CSV format and upload that CSV file to Expedition for processing.
What is the process to manually upload the firewall logs onto Expedition?
Make sure that you are exporting the logs to a folder that exists within Expedition.
Personally, I would recommend to use the path /PALogs, which you would have to create giving the following commands:
chown expedition:www-data /PALogs
chmod 750 /PALogs
With those, "expedition", the user that you use to send logs via sftp, will be able to write into the folder, and www-data, who reads the logs for generating the connections.parquet (CSV preprocessing) and RE and ML will have read rights.
Uploading the files manually only requires you to establish a SCP connection and place them in the path that you desire.
The path should be reacheable by www-data (meaning that the folders towards the complete path should be executable by www-data) and file should be visible/readeable by www-data.
Again, I would suggest placing them into /PALogs. You may desire to create subfolders for each FW, but this step is not necessary as we inspect the CSV content to identify the device's serial number.
Thanks, we've followed the instructions provided and created folder PALogs and transferred the log there. We've also added the firewall as per instructions in page 16 of Expedition.
We are unable to find tab Plugins to continue with page 17. Has this tab been removed from v1.1.12?
How can be benefit from the ML analysis of these logs done by Expedition? An Excel doc has been created in Traffic tab including:
I guess that you are mixing two steps from the documentation.
1) Plugins are used within a Project to define the sources from where do you want to apply the learning.
2) Another prior step is to prepare the source, which it is to load the CSV logs into the Expedition folder, and preprocess them to make them ready for ML.
If you have tested that the path is correct (I assume we are talking about the "Test SCP server connection" option that the NGFW have), this means that you have the permissions correct in your Expedition to receive the logs.
If you do not receive logs, my idea would be that maybe you still need to Commit in the Firewall the change regarding performing the Schedule log export.
Another option is to download the logs via the "Export to CSV" in the Monitor tab and upload them to the desired folder in Expedition via SCP.
This is not a desired way, as it requires manual upload each time you want to check logs, but if this a puntual task that you want to do, it may be fine.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!