Expedition Unable to Import Logs from PA-1410

[DanaHawkins]

I have a customer that is deploying a brand new PA-1410. This site is a greenfield site so they want to build security policies as they bring devices online. They've recently provided me with an export of logs and I'm trying to get them imported into Expedition. I want to run ML against the logs to build a base policy set based on what's seen. Then also provided me a configuration export as well which I'm using as my base configuration in Expedition.

No matter what I do I can't get the logs to show up for processing. I even went as far as reinstalling Expedition from scratch.

Can anyone tell me if PanOS 11.0 is supported in Expedition yet?

[lychiang]

Hi @DanaHawkins  Yes, PAN-OS 11.x should be supported, few things to check :

1. Make sure your expedition is upgraded to v1.2.59

2. Review ML settings are properly defined , the ML address match the expedition IP, if you don't know what's the IP, you can type 127.0.0.1 and click "save", it will show you the correct ML IP. Make sure the /data are listed in connection parquet settings:

3. Review ML folder permission /data and /PALlogs  like below:

if the permissions are not correct , you will need to issue below commands:

#sudo chown -R www-data:www-data /data /PALogs

#sudo chmod -R 775 /data /PALogs 

4. Make sure you are processing the logs on the correct Firewall device , the traffic log contain the serial# of the device that needs to match the serial # of the the firewall device

5. When process the logs, you can review below error logs:

1. /home/userSpace/panReadOrders.log: Review the call with the params for the spark process.
2. /tmp/command.spark: Review the call to spark. It can be copied and executed by hand for troubleshooting purposes, output will be printed on the cli.
3. /tmp/command_actions.spark: Review the call to spark. It can be copied and executed by hand for troubleshooting purposes, output will be printed on /tmp/error_logCoCo.
4. /tmp/error_logCoCo: File containing the output of the spark command.

[DanaHawkins]

Thanks for the info. I'm going to review the logs to see if I can find anything in there that might help. I created the firewall device with the same serial number as what's in the logs. The only thing I can think of is 1400 series isn't a choice when you create the device. I just chose VM-Series.

I will let you know what I find.

[DanaHawkins]

I'm going to have the customer re-export the log files again.

Checking: PeriodicLogCollectorCompacter
Mon, 01 May 2023 14:19:05 -0700 Start Task
Checking CSV logs from device(s) 2670100XXXX
Checking ML server is alive
ML Server is alive
Collecting device(s) serial(s)
2670100XXXX added
devicesData
Array
(
[0] => stdClass Object
(
[serial] => 2670100XXXX
[afterProcess] =>
)

)
serialsAndData
Array
(
[0] => Array
(
[serial] => 2670100XXXX
[path] => /PALogs/firewall/*
)

)
No supported new files to process

Success: -1 Errors: No supported new files to process

Mon, 01 May 2023 14:19:05 -0700 End Task

[DanaHawkins]

I'm still running into the same issue as I stated above. I loaded expedition on a completely different system as well. Has there been a change in the log format from 10.X to 11.X?

expedition@del-expedition01:/PALogs/firewall$ sudo tail -f /home/userSpace/panReadOrders.log

)
No supported new files to process

Success: -1 Errors: No supported new files to process

Tue, 02 May 2023 12:02:47 -0700 End Task

Tue, 02 May 2023 12:05:17 -0700 Start Task
Checking CSV logs from device(s) 267010XXXXX
Checking ML server is alive
ML Server is alive
Collecting device(s) serial(s)
267010XXXXX added
devicesData
Array
(
[0] => stdClass Object
(
[serial] => 267010XXXXX
[afterProcess] =>
)

)
serialsAndData
Array
(
[0] => Array
(
[serial] => 267010XXXXX
[path] => /PALogs/firewall/*
)

)
No supported new files to process

Success: -1 Errors: No supported new files to process

Tue, 02 May 2023 12:05:17 -0700 End Task

[lychiang]

Hi Dana,

I see the path is  /PALogs/firewall/* , can you please make sure the traffic logs are directly saved in /PALogs folder without any subfolder , also make sure all traffic logs under /PALogs have same permissions and owned by www-data 

[DanaHawkins]

I moved the csv file as you mentioned above.

I also updated the ML configuration as well.

I pulled the serial number directly from the log to create the device in Expedition. Expedition itself doesn't have access to the device though as it sit on the customer's network.

[lychiang]

You will need direct connection from expedition to firewall to be able to retrieve running configuration

[DanaHawkins]

The customer exported the running configuration and provided it to me and I uploaded it manually. Expedition took the files without issue and I was able to import the device configuration into the project.