Palo Alto Expedition - Machine Learning Process Enabled Files RED Enabled

Reply
L0 Member

Palo Alto Expedition - Machine Learning Process Enabled Files RED Enabled

Expedition 1.1.89

 

running machine learning on the traffic logs.

 

I've given www-data full permissions and chmod 777 to the file /data and all .csv files

 

When I try to "Process Files" under Palo Alto Network Devices>M.learning all i see in the comments is "Today's file in progress"

 

exp3.JPGexp2.JPGexp1.JPG


Accepted Solutions
L5 Sessionator

According on how you have defined the file (Log files come from Syslog), today's file will not be processed.

This is because today's file is not over yet, new traffic logs are still coming and will remain coming until 23:59:59.999.

So, Expedition will prevent you from processing today's file so you do not miss, in your processing, data that would be later valuable for doing Rule Enrichment or Machine Learning.

 

As I see, you have defined in Expedition that you want this Firewall's logs to be automatically processed at 5:57:30, so, tomorrow at this time, this log (that by tomorrow it will be yesterday's log) will be processed.

 

One thing to mention regarding the file permissions. If you can see the file listed in the grid, it means that you have, at least, correct reading permissions to the file. This would be enough, unless you also want to be able to delete/compress after processing. In such case, you also want writing rights on the file.

Giving 777 permissions on files may be excessive. Normally your file belongs to the user "expedition" and the group "expedition". www-data if I am not mistaken, is part of the "expedition" user group. Therefore, with a 660 should be enough.

Remember that this translates to

rw-rw----
meaning:
rw- "the user expedition can read and write, but not execute (however, we do not execute CSV files)"

rw- "the group expedition can read and write the file"

--- the rest of users do not have access to the file

 

Notice also that we need to be able to reach the file itself. In your case this is within the /data folder. Therefore, the expedition and www-data users should be able to enter (execute) that folder.

So, for the folder we may want to provide a 770 to www-data:expedition. If we would have the files in further nested folders, we should provide access to those folders as well.

View solution in original post

Tags (1)

All Replies
L5 Sessionator

According on how you have defined the file (Log files come from Syslog), today's file will not be processed.

This is because today's file is not over yet, new traffic logs are still coming and will remain coming until 23:59:59.999.

So, Expedition will prevent you from processing today's file so you do not miss, in your processing, data that would be later valuable for doing Rule Enrichment or Machine Learning.

 

As I see, you have defined in Expedition that you want this Firewall's logs to be automatically processed at 5:57:30, so, tomorrow at this time, this log (that by tomorrow it will be yesterday's log) will be processed.

 

One thing to mention regarding the file permissions. If you can see the file listed in the grid, it means that you have, at least, correct reading permissions to the file. This would be enough, unless you also want to be able to delete/compress after processing. In such case, you also want writing rights on the file.

Giving 777 permissions on files may be excessive. Normally your file belongs to the user "expedition" and the group "expedition". www-data if I am not mistaken, is part of the "expedition" user group. Therefore, with a 660 should be enough.

Remember that this translates to

rw-rw----
meaning:
rw- "the user expedition can read and write, but not execute (however, we do not execute CSV files)"

rw- "the group expedition can read and write the file"

--- the rest of users do not have access to the file

 

Notice also that we need to be able to reach the file itself. In your case this is within the /data folder. Therefore, the expedition and www-data users should be able to enter (execute) that folder.

So, for the folder we may want to provide a 770 to www-data:expedition. If we would have the files in further nested folders, we should provide access to those folders as well.

View solution in original post

Tags (1)
L0 Member

thank you for the explanation. The scheduled traffic logs did come in later and was processed.

 

Thank you for the alternative to chmod 777

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!