topic Re: Panorama logs missing beofre a date in General Topics

Panorama logs missing beofre a date

soporteseguridad — Tue, 16 Feb 2016 08:25:35 GMT

Hi,

We have 2 FWs sending logs to Panorama. We see the logs in Panorama after 3 January but not the logs previous this date. Why??? we havent done anything in Panotama for not to see this previous logs.

Regards,

Re: Panorama logs missing beofre a date

reaper — Tue, 16 Feb 2016 09:54:26 GMT

Have you verified there's still disk space available for logs to be retained longer than before that dat ?

> show system disk-space 

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             7.6G  1.5G  5.7G  21% /
/dev/sda5              23G  6.4G   16G  30% /opt/pancfg
/dev/sda6              16G  4.6G  9.8G  32% /opt/panrepo
tmpfs                 7.9G     0  7.9G   0% /dev/shm
/dev/sda8              56G   17G   36G  32% /opt/panlogs
/dev/loop0             16G  173M   15G   2% /opt/logbuffer
/dev/md1              917G   29G  842G   4% /opt/panlogs/ld1

> show system logdb-quota 

Quotas:
              system: 8.00%, 3.243 GB Expiration-period: 0 days
              config: 8.00%, 3.243 GB Expiration-period: 0 days
             appstat: 5.00%, 2.027 GB Expiration-period: 0 days

Disk usage:
system: Logs and Indexes: 114.9MB Current Retention: 302 days
config: Logs and Indexes: 154.7MB Current Retention: 299 days
appstatdb: Logs and Indexes: 64.5MB Current Retention: 302 days

Slot:0
        Quotas:
                traffic: 25.00%, 207 GB Expiration-period: 0 days
                threat: 25.00%, 207 GB Expiration-period: 0 days
                system: 8.00%, 66 GB Expiration-period: 0 days
...

        Disk usage:
                traffic: Logs and Indexes: 4580 MB Current Retention: 328 days
                threat: Logs and Indexes: 268 MB Current Retention: 305 days
                system: Logs and Indexes: 292 MB Current Retention: 374 days
...

Re: Panorama logs missing beofre a date

soporteseguridad — Tue, 16 Feb 2016 11:47:13 GMT

Hi,

I attach the output. It seems like its not a size HDD problem.

admin@Panorama> show system disk-space

Filesystem Size Used Avail Use% Mounted on
/dev/sda2 3.8G 1.4G 2.2G 40% /
/dev/sda5 23G 3.9G 18G 19% /opt/pancfg
/dev/sda6 3.8G 2.0G 1.6G 56% /opt/panrepo
tmpfs 3.0G 0 3.0G 0% /dev/shm
/dev/sda8 2.9T 804G 2.1T 28% /opt/panlogs
10.10.30.1:/vol/fich_red_logs_SATA_vol/panorama
2.9T 804G 2.1T 28% /mnt/dynamic-logs

admin@Panorama>

admin@Panorama> show system logdb-quota

Quotas:
system: 5.00%, 145.595 GB Expiration-period: 0 days
config: 3.00%, 87.357 GB Expiration-period: 0 days
appstat: 10.00%, 291.190 GB Expiration-period: 0 days
traffic: 24.00%, 698.856 GB Expiration-period: 0 days
threat: 29.00%, 844.451 GB Expiration-period: 0 days
trsum: 5.00%, 145.595 GB Expiration-period: 0 days
hourlytrsum: 1.00%, 29.119 GB Expiration-period: 0 days
dailytrsum: 1.00%, 29.119 GB Expiration-period: 0 days
weeklytrsum: 1.00%, 29.119 GB Expiration-period: 0 days
urlsum: 3.00%, 87.357 GB Expiration-period: 0 days
hourlyurlsum: 1.00%, 29.119 GB Expiration-period: 0 days
dailyurlsum: 1.00%, 29.119 GB Expiration-period: 0 days
weeklyurlsum: 1.00%, 29.119 GB Expiration-period: 0 days
thsum: 5.00%, 145.595 GB Expiration-period: 0 days
hourlythsum: 1.00%, 29.119 GB Expiration-period: 0 days
dailythsum: 1.00%, 29.119 GB Expiration-period: 0 days
weeklythsum: 1.00%, 29.119 GB Expiration-period: 0 days
extpcap: 1.00%, 29.119 GB Expiration-period: 0 days
hipmatch: 1.00%, 29.119 GB Expiration-period: 0 days

Disk usage:
traffic: Logs and Indexes: 688.2GB Current Retention: 44 days
threat: Logs and Indexes: 6.9GB Current Retention: 361 days
system: Logs and Indexes: 451.7MB Current Retention: 637 days
config: Logs and Indexes: 572.3MB Current Retention: 637 days
trsum: Logs and Indexes: 145.5GB Current Retention: 73 days
hourlytrsum: Logs and Indexes: 29.1GB Current Retention: 25 days
dailytrsum: Logs and Indexes: 13.3GB Current Retention: 260 days
weeklytrsum: Logs and Indexes: 2.1GB Current Retention: 276 days
thsum: Logs and Indexes: 2.4GB Current Retention: 354 days
hourlythsum: Logs and Indexes: 1.9GB Current Retention: 263 days
dailythsum: Logs and Indexes: 732.7MB Current Retention: 310 days
weeklythsum: Logs and Indexes: 594.8MB Current Retention: 353 days
appstatdb: Logs and Indexes: 1.9GB Current Retention: 637 days
hipmatch: Logs and Indexes: 0 Current Retention: 0 days
extpcap: Logs and Indexes: 1.3GB Current Retention: 279 days
urlsum: Logs and Indexes: 180.0KB Current Retention: 0 days
hourlyurlsum: Logs and Indexes: 168.0KB Current Retention: 0 days
dailyurlsum: Logs and Indexes: 104.0KB Current Retention: 0 days
weeklyurlsum: Logs and Indexes: 16.0KB Current Retention: 0 days

Re: Panorama logs missing beofre a date

soporteseguridad — Tue, 16 Feb 2016 12:08:37 GMT

where can i change the retention date for more days?

Re: Panorama logs missing beofre a date

Raido_Rattameister — Tue, 16 Feb 2016 13:07:13 GMT

Panorama > Setup > Management > Logging and Reporting Settings

Re: Panorama logs missing beofre a date

soporteseguridad — Wed, 17 Feb 2016 12:07:10 GMT

We have several doubts.

We upgrade from version 6.1.5 to 7.0.4. And now the traffic log max is 44 days. Why is 44 days if we didnt touch this value??? and the default valores is infinite.

this max retention days was in 6.1.x version or only in 7.0.4???

regards,

Re: Panorama logs missing beofre a date

reaper — Wed, 17 Feb 2016 13:24:20 GMT

this is not the hard-set retention days, but the estimated retention based on the influx of logs and the storage available

admin@Panorama> show system logdb-quota

Quotas:
...
traffic: 24.00%, 698.856 GB Expiration-period: 0 days
...

Disk usage:
traffic: Logs and Indexes: 688.2GB Current Retention: 44 days
...

you have assigned 24% of 2.9tb = +-698 gb of space to the traffic log (but you still have a little bit of indexing overhead)

it is currently filled with 688gb of logs and estimated retention is 44 days, meaning it is receiving logs at such a rate, that 45 day old logs are being deleted to make room for new log

Re: Panorama logs missing beofre a date

Lucky — Wed, 17 Feb 2016 21:29:48 GMT

Hi, Soport Seguridad,

for what it's worth, change in behavior was noted in release notes for 7.0, in "Management Features", with a link for more information. I know this is "captain hindsight" help 😕

Reaper is right with his answer too, this is only a calculation based on your current situation.

You can check defaults by clicking "restore defaults" in the bottom right corner of the panel for logging and reporting settings, to do that go to Device > Setup > Management > Logging and Reporting Settings, as in screenshot.

Best regards,

Luciano