01-04-2019 03:38 PM
Hi Team!
We wish to use the Expedition tool for some logs coming from a PA-7000 series. Since the scheduled log export option is not feasible we wish to export logs via syslog. A few questions with this.
1. Where is the default location that the syslogs will go to?
2. Can we change the default log location, if so how? (We will be adding new virtual disk mounted to /PALogs and want to make the logs go there)
3. What will the permissions need to be on the new logs folder, does it need to be owned by "syslog" or "expedition", both? How?
I have a funny feeling its to do with the rsyslog.confg file but could do with some assistance.
Cheers,
Luke.
01-09-2019 03:06 AM - edited 01-11-2019 02:56 AM
Hi Luke,
Yes, Expedition can get the logs as a syslog server as well.
In the /etc/rsyslog.conf you would define how the syslog server (RSyslog in Expedition case) would process the received entries:
As we would like to help you speeding up the process of setting up the syslog server, we have provided three useful examples that you can simply take for your Expedition instance.
You can find them inside the folder
/var/www/html/OS/rsyslog/
Inside, you will find a config to listen only for UDP connections, for TCP connections or both.
Take the one you prefer, edit it to add the IPs inside the AllowedSender parameteryou would allow to send syslog entries (tipically the IPs for the reporting interfaces in the PA7000) (by default shows:)
$AllowedSender TCP, 127.0.0.1, 10.11.29.0/24, 172.16.26.0/24, *.paloaltonetworks.com
and save it into /etc with the name rsyslog.conf
Remember to restart the syslog service by
service rsyslog restart
We have designed the config to save the logs into a /data folder that should exist (create it if it doesn't) and are stored by reporting IP and day (we try to emulate the structure that a firewall would create with the log export).
Regarding the permits, thing that the "rsyslog" group will store the logs there, but that www-data needs to be able to read them as well to generate the parquets and perform the ML activities.
$template DynaTrafficLog,"/data/%FROMHOST-IP%/%HOSTNAME%_traffic_%$YEAR%_%$MONTH%_%$DAY%_last_calendar_day.csv"
Finally, make sure that your Expedition instance is allowed to listen the desired port. Expedition has firewalld service that you may need to edit it. For instance:
sudo /usr/bin/firewall-cmd --permanent --add-port=514/udp
Let's start with this.
UPDATE: We have noticed that in some situations, it may be required to restart the machine to ensure that rsyslog captures the arriving traffic and process it.
01-09-2019 03:06 AM - edited 01-11-2019 02:56 AM
Hi Luke,
Yes, Expedition can get the logs as a syslog server as well.
In the /etc/rsyslog.conf you would define how the syslog server (RSyslog in Expedition case) would process the received entries:
As we would like to help you speeding up the process of setting up the syslog server, we have provided three useful examples that you can simply take for your Expedition instance.
You can find them inside the folder
/var/www/html/OS/rsyslog/
Inside, you will find a config to listen only for UDP connections, for TCP connections or both.
Take the one you prefer, edit it to add the IPs inside the AllowedSender parameteryou would allow to send syslog entries (tipically the IPs for the reporting interfaces in the PA7000) (by default shows:)
$AllowedSender TCP, 127.0.0.1, 10.11.29.0/24, 172.16.26.0/24, *.paloaltonetworks.com
and save it into /etc with the name rsyslog.conf
Remember to restart the syslog service by
service rsyslog restart
We have designed the config to save the logs into a /data folder that should exist (create it if it doesn't) and are stored by reporting IP and day (we try to emulate the structure that a firewall would create with the log export).
Regarding the permits, thing that the "rsyslog" group will store the logs there, but that www-data needs to be able to read them as well to generate the parquets and perform the ML activities.
$template DynaTrafficLog,"/data/%FROMHOST-IP%/%HOSTNAME%_traffic_%$YEAR%_%$MONTH%_%$DAY%_last_calendar_day.csv"
Finally, make sure that your Expedition instance is allowed to listen the desired port. Expedition has firewalld service that you may need to edit it. For instance:
sudo /usr/bin/firewall-cmd --permanent --add-port=514/udp
Let's start with this.
UPDATE: We have noticed that in some situations, it may be required to restart the machine to ensure that rsyslog captures the arriving traffic and process it.
01-10-2019 03:40 AM
Hi @dgildelaig
We attempted this today with unfortunately no luck.
We modified the rsyslog-default-udp config file and then copied it to /etc/ as rsyslog.conf (we changed the rsyslog.conf that already existed there as rsyslog.conf.bak)
Unfortunately there are no logs coming into the folder we specified. FW rule was created and syslog has permissions to the folder, restarted syslog service.
We then copy pasted the snippet from the default-udp file into the already existing rsyslog.conf file in the /etc/ folder, restarted syslog service. We now have the "last calandar day.csv" file created in our log folder but it only contains rsyslog log files rather than FW log files. Please help! 🙂
04-04-2019 11:00 PM
Do i need to do anything else for Expedition to automatically parse those files?
Config works.
root@Expedition:/data/10.127.2.XX# ls -al
total 28496
drwx------ 2 root root 4096 Apr 5 00:37 .
drwxr-xr-x 7 syslog root 4096 Apr 5 00:37 ..
-rw-r--r-- 1 root root 29170806 Apr 5 00:58 xxx.xxxxxxx.xxx_traffic_2019_04_05_last_calendar_day.csv
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!