Palo Alto Expedition - Machine Learning Process Enabled Files RED Enabled

[jasonwa]

Expedition 1.1.89

running machine learning on the traffic logs.

I've given www-data full permissions and chmod 777 to the file /data and all .csv files

When I try to "Process Files" under Palo Alto Network Devices>M.learning all i see in the comments is "Today's file in progress"

[dgildelaig · Accepted solution]

According on how you have defined the file (Log files come from Syslog), today's file will not be processed.

This is because today's file is not over yet, new traffic logs are still coming and will remain coming until 23:59:59.999.

So, Expedition will prevent you from processing today's file so you do not miss, in your processing, data that would be later valuable for doing Rule Enrichment or Machine Learning.

As I see, you have defined in Expedition that you want this Firewall's logs to be automatically processed at 5:57:30, so, tomorrow at this time, this log (that by tomorrow it will be yesterday's log) will be processed.

One thing to mention regarding the file permissions. If you can see the file listed in the grid, it means that you have, at least, correct reading permissions to the file. This would be enough, unless you also want to be able to delete/compress after processing. In such case, you also want writing rights on the file.

Giving 777 permissions on files may be excessive. Normally your file belongs to the user "expedition" and the group "expedition". www-data if I am not mistaken, is part of the "expedition" user group. Therefore, with a 660 should be enough.

Remember that this translates to

rw-rw----
meaning:
rw- "the user expedition can read and write, but not execute (however, we do not execute CSV files)"

rw- "the group expedition can read and write the file"

--- the rest of users do not have access to the file

Notice also that we need to be able to reach the file itself. In your case this is within the /data folder. Therefore, the expedition and www-data users should be able to enter (execute) that folder.

So, for the folder we may want to provide a 770 to www-data:expedition. If we would have the files in further nested folders, we should provide access to those folders as well.

[jasonwa]

thank you for the explanation. The scheduled traffic logs did come in later and was processed.

Thank you for the alternative to chmod 777