Expedition Discussions

Reply
L5 Sessionator
Posts: 221
Registered: ‎12-08-2016
Accepted Solution

Expedition as syslog server, change logs directory

Hi Team!

 

We wish to use the Expedition tool for some logs coming from a PA-7000 series. Since the scheduled log export option is not feasible we wish to export logs via syslog. A few questions with this.

 

1. Where is the default location that the syslogs will go to?

2. Can we change the default log location, if so how? (We will be adding new virtual disk mounted to /PALogs and want to make the logs go there)

3. What will the permissions need to be on the new logs folder, does it need to be owned by "syslog" or "expedition", both? How?

 

I have a funny feeling its to do with the rsyslog.confg file but could do with some assistance.

 

Cheers,

Luke.

L4 Transporter
Posts: 179
Registered: ‎11-02-2015

Re: Expedition as syslog server, change logs directory

[ Edited ]

Hi Luke,

 

Yes, Expedition can get the logs as a syslog server as well.

In the /etc/rsyslog.conf you would define how the syslog server (RSyslog in Expedition case) would process the received entries:

  • where to listen from (notice that the example defined the 10514 port, which it is not the default one. Probably you would prefer setting it up to 514)
  • which IPs to allow reporting
  • where to store the log entries

As we would like to help you speeding up the process of setting up the syslog server, we have provided three useful examples that you can simply take for your Expedition instance.

You can find them inside the folder 

/var/www/html/OS/rsyslog/

 

Inside, you will find a config to listen only for UDP connections, for TCP connections or both.

Take the one you prefer, edit it to add the IPs inside the AllowedSender parameteryou would allow to send syslog entries (tipically the IPs for the reporting interfaces in the PA7000) (by default shows:)

$AllowedSender TCP, 127.0.0.1, 10.11.29.0/24, 172.16.26.0/24, *.paloaltonetworks.com

and save it into /etc with the name rsyslog.conf

 

Remember to restart the syslog service by

service rsyslog restart

 

We have designed the config to save the logs into a /data folder that should exist (create it if it doesn't) and are stored by reporting IP and day (we try to emulate the structure that a firewall would create with the log export).

Regarding the permits, thing that the "rsyslog" group will store the logs there, but that www-data needs to be able to read them as well to generate the parquets and perform the ML activities.

$template DynaTrafficLog,"/data/%FROMHOST-IP%/%HOSTNAME%_traffic_%$YEAR%_%$MONTH%_%$DAY%_last_calendar_day.csv"

 

Finally, make sure that your Expedition instance is allowed to listen the desired port. Expedition has firewalld service that you may need to edit it. For instance:

sudo /usr/bin/firewall-cmd --permanent --add-port=514/udp

 

Let's start with this.

 

UPDATE: We have noticed that in some situations, it may be required to restart the machine to ensure that rsyslog captures the arriving traffic and process it.

Highlighted
L5 Sessionator
Posts: 221
Registered: ‎12-08-2016

Re: Expedition as syslog server, change logs directory

Many thanks, @dgildelaig!

 

I will let you know how I get on with this.

L5 Sessionator
Posts: 221
Registered: ‎12-08-2016

Re: Expedition as syslog server, change logs directory

Hi @dgildelaig


We attempted this today with unfortunately no luck.

 

We modified the rsyslog-default-udp config file and then copied it to /etc/  as rsyslog.conf (we changed the rsyslog.conf that already existed there as rsyslog.conf.bak) 

 

Unfortunately there are no logs coming into the folder we specified. FW rule was created and syslog has permissions to the folder, restarted syslog service.

 

We then copy pasted the snippet from the default-udp file into the already existing rsyslog.conf file in the /etc/ folder, restarted syslog service. We now have the "last calandar day.csv" file created in our log folder but it only contains rsyslog log files rather than FW log files. Please help! :)

L2 Linker
Posts: 12
Registered: ‎05-30-2016

Re: Expedition as syslog server, change logs directory

Do i need to do anything else for Expedition to automatically parse those files?

 

Config works.

 

root@Expedition:/data/10.127.2.XX# ls -al

total 28496

drwx------ 2 root   root     4096 Apr  5 00:37 .

drwxr-xr-x 7 syslog root     4096 Apr  5 00:37 ..

-rw-r--r-- 1 root   root 29170806 Apr  5 00:58 xxx.xxxxxxx.xxx_traffic_2019_04_05_last_calendar_day.csv

L4 Transporter
Posts: 179
Registered: ‎11-02-2015

Re: Expedition as syslog server, change logs directory

For anyone reading this thread in the future, we have found that in some cases, to see logs getting into the desired folder, it is ALSO necessary to restart the VM. Restarting the service did not seem to be enough, and as I am not a guru to identify which other services are involved in this process, what we found out is that restarting the VM helped to start seeing the logs arriving into the desired folder.

 

We spent hours with a client to see whether the config was wrong (which look perfect), sniffing traffic to see that was hitting the Expedition VM from the Firewalls that were forwarding the syslog entries, and at the end, we just reboot the VM and voilá!

 

I hope this helps others.

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!