Logs import for ML

[ash83]

We are following Expedition-LogAnalysisGuide_v1.0.2.pdf to start using ML  

Expedition is v1.1.12, BP v3.6.3, Task Manager is running and there are no internal checks to remediate

The firewalls are VMs running 9.0.1

In page 6 in the above doc, IP address 192.168.44.131 is used for Expedition. However, in page 7, a different IP is used in section Hostname: 10.30.11.50

Please confirm if the IP to configure in Schedule Log Report-Hostname should be the IP for Expedition.

We worked under this assumption, but we get the following error when clicking on Test SCP server connection: Error response from server: bash: /home/expedition/logs/ssh-export-test.txt: No such file or directory

What is causing this error and how can it be fixed?

Until the above is sorted, we would like to upload the logs manually, but the pdf does not include instructions on how to accomplish this:

MANUALLY EXPORT LOGS FROM MONITOR

You can always go to any firewall from Palo Alto Networks and from the Monitor tab export the logs in CSV format and upload that CSV file to Expedition for processing.

What is the process to manually upload the firewall logs onto Expedition?

Thanks.

Ho

[ash83]

Anyone?? Thanks

Ho

[dgildelaig]

Make sure that you are exporting the logs to a folder that exists within Expedition.

Personally, I would recommend to use the path /PALogs, which you would have to create giving the following commands:

mkdir /PALogs
chown expedition:www-data /PALogs

chmod 750 /PALogs

With those, "expedition", the user that you use to send logs via sftp, will be able to write into the folder, and www-data, who reads the logs for generating the connections.parquet (CSV preprocessing) and RE and ML will have read rights.

Uploading the files manually only requires you to establish a SCP connection and place them in the path that you desire.

The path should be reacheable by www-data (meaning that the folders towards the complete path should be executable by www-data) and file should be visible/readeable by www-data.

Again, I would suggest placing them into /PALogs. You may desire to create subfolders for each FW, but this step is not necessary as we inspect the CSV content to identify the device's serial number.

[ash83]

Thanks, we've followed the instructions provided and created folder PALogs and transferred the log there. We've also added the firewall as per instructions in page 16 of Expedition.

We are unable to find tab Plugins to continue with page 17. Has this tab been removed from v1.1.12?

How can be benefit from the ML analysis of these logs done by Expedition? An Excel doc has been created in Traffic tab including:

Device

Vsys

Date

Rule Name

Typology

Hits

Thanks.

Ho

[ash83]

Hi, wondering if anyone could assist with the last question?

Thanks.

Ho