Expedition as syslog server, change logs directory

[LukeBullimore]

Hi Team!

We wish to use the Expedition tool for some logs coming from a PA-7000 series. Since the scheduled log export option is not feasible we wish to export logs via syslog. A few questions with this.

1. Where is the default location that the syslogs will go to?

2. Can we change the default log location, if so how? (We will be adding new virtual disk mounted to /PALogs and want to make the logs go there)

3. What will the permissions need to be on the new logs folder, does it need to be owned by "syslog" or "expedition", both? How?

I have a funny feeling its to do with the rsyslog.confg file but could do with some assistance.

Cheers,

Luke.

[dgildelaig · Accepted solution]

Hi Luke,

Yes, Expedition can get the logs as a syslog server as well.

In the /etc/rsyslog.conf you would define how the syslog server (RSyslog in Expedition case) would process the received entries:

• where to listen from (notice that the example defined the 10514 port, which it is not the default one. Probably you would prefer setting it up to 514)
• which IPs to allow reporting
• where to store the log entries

As we would like to help you speeding up the process of setting up the syslog server, we have provided three useful examples that you can simply take for your Expedition instance.

You can find them inside the folder 

/var/www/html/OS/rsyslog/

Inside, you will find a config to listen only for UDP connections, for TCP connections or both.

Take the one you prefer, edit it to add the IPs inside the AllowedSender parameteryou would allow to send syslog entries (tipically the IPs for the reporting interfaces in the PA7000) (by default shows:)

$AllowedSender TCP, 127.0.0.1, 10.11.29.0/24, 172.16.26.0/24, *.paloaltonetworks.com

and save it into /etc with the name rsyslog.conf

Remember to restart the syslog service by

service rsyslog restart

We have designed the config to save the logs into a /data folder that should exist (create it if it doesn't) and are stored by reporting IP and day (we try to emulate the structure that a firewall would create with the log export).

Regarding the permits, thing that the "rsyslog" group will store the logs there, but that www-data needs to be able to read them as well to generate the parquets and perform the ML activities.

$template DynaTrafficLog,"/data/%FROMHOST-IP%/%HOSTNAME%_traffic_%$YEAR%_%$MONTH%_%$DAY%_last_calendar_day.csv"

Finally, make sure that your Expedition instance is allowed to listen the desired port. Expedition has firewalld service that you may need to edit it. For instance:

sudo /usr/bin/firewall-cmd --permanent --add-port=514/udp

Let's start with this.

UPDATE: We have noticed that in some situations, it may be required to restart the machine to ensure that rsyslog captures the arriving traffic and process it.

[LukeBullimore]

Many thanks, @dgildelaig!

I will let you know how I get on with this.

[LukeBullimore]

Hi @dgildelaig

We attempted this today with unfortunately no luck.

We modified the rsyslog-default-udp config file and then copied it to /etc/  as rsyslog.conf (we changed the rsyslog.conf that already existed there as rsyslog.conf.bak) 

Unfortunately there are no logs coming into the folder we specified. FW rule was created and syslog has permissions to the folder, restarted syslog service.

We then copy pasted the snippet from the default-udp file into the already existing rsyslog.conf file in the /etc/ folder, restarted syslog service. We now have the "last calandar day.csv" file created in our log folder but it only contains rsyslog log files rather than FW log files. Please help! 🙂

[Laimonas]

Do i need to do anything else for Expedition to automatically parse those files?

Config works.

root@Expedition:/data/10.127.2.XX# ls -al

total 28496

drwx------ 2 root   root     4096 Apr  5 00:37 .

drwxr-xr-x 7 syslog root     4096 Apr  5 00:37 ..

-rw-r--r-- 1 root   root 29170806 Apr  5 00:58 xxx.xxxxxxx.xxx_traffic_2019_04_05_last_calendar_day.csv