topic Re: EXPEDITION : ML is not able to Parse connection logs in Expedition Discussions

EXPEDITION : ML is not able to Parse connection logs

melvyn-guibout — Wed, 13 Mar 2024 14:24:09 GMT

Hello

I am not able to use the ML functionnality on imported log trafic.

Spark tasks are skipped..

I have nothing in those directories :

Log files are in PALogs/

PALogs directory is owned by www-data.

Here is my ML menu when i try to analyse data :

(nothing unusual)

But there is nothing in the output.

How can i pinpoint the problem more precisely since i have no error logs ?

Thank you,

Melvyn

Re: EXPEDITION : ML is not able to Parse connection logs

dpuigdomenec — Wed, 13 Mar 2024 14:33:40 GMT

Hi @melvyn-guibout

Let me suggest some troubleshooting:

1) Check that the logs are analysed so you have a folder /data/ with the parquet files. That folder shall be owned by www-data:www-data.

2) Check that the serial displayed in the "Connectors" match with the serial on the analysed logs.

3) Check the generated logs in /tmp/ folder.

ML Logs

File	Content
/tmp/command.spark	External cli command to execute spark
/tmp/error_SecRulesLearn	Standard output execution log for ML process
/tmp/error_SecRulesLearn2	Error output execution log for ML process

RE Logs

File	Content
/tmp/command.spark	External cli command to execute spark
/tmp/error_SecRulesEnrich	Standard output execution log for RE process
/tmp/error_SecRulesEnrich2	Error output execution log for RE process

Let me know if you need anything else,

Best,

Re: EXPEDITION : ML is not able to Parse connection logs

C.Pfitzer — Mon, 18 Mar 2024 17:29:07 GMT

You may find hints in /tmp/error_SecRulesLearn.

Our Panorama is M300, which is not available in the drop down menu, so I gave it a try with M600, but no output. We had to change the model type to VM, then it worked.

Re: EXPEDITION : ML is not able to Parse connection logs

dpuigdomenec — Mon, 18 Mar 2024 18:45:50 GMT

Hi @C.Pfitzer, @melvyn-guibout

To be able to create properly the log connector, if the configuration is from a Panorama, on the device model select vm-panorama.

Thanks!

David