topic Re: log retention days in Panorama Discussions

log retention days

JeffKim — Wed, 23 Mar 2022 21:09:50 GMT

Hi we have 2 panorama and it has virtual disks for log-collector.

I have checked log-collector-es-cluster health and it is green.

1 collector-group and 2 log-collectors

When i run cli

show system logdb-quota at the active panorama , i get result as below

How can I understand expiration-period is 30 days , but I can't see more than 16 days

Is it disk volume issue ? IMHO , it looks overwrite traffic log older than 16 days

Quotas:
system: 8.00%, 1.072 GB Expiration-period: 7 days
config: 8.00%, 1.072 GB Expiration-period: 7 days
hip-reports: 1.00%, 0.134 GB Expiration-period: 0 days
appstat: 5.00%, 0.670 GB Expiration-period: 0 days

Disk usage:
system: Logs and Indexes: 844.9MB Current Retention: 7 days
config: Logs and Indexes: 28.8MB Current Retention: 7 days
appstatdb: Logs and Indexes: 691.5MB Current Retention: 20 days
hip-reports: Logs and Indexes: 0 Current Retention: 0 days

Slot:0
Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
summary: 30.00%, 141 GB Expiration-period: 30 days
infra_audit: 5.00%, 24 GB Expiration-period: 30 days
platform: 0.10%, 0 GB Expiration-period: 30 days
external: 0.10%, 0 GB Expiration-period: 30 days

Disk usage:
detailed: Logs: 137161 MB, Current Retention: 14 days
summary: Logs: 21456 MB, Current Retention: 27 days
infra_audit: Logs: 1425 MB, Current Retention: 21 days
platform: Logs: 0 MB, Current Retention: 0 days
external: Logs: 0 MB, Current Retention: 0 days

Slot:1
Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
summary: 30.00%, 141 GB Expiration-period: 30 days
infra_audit: 5.00%, 24 GB Expiration-period: 30 days
platform: 0.10%, 0 GB Expiration-period: 30 days
external: 0.10%, 0 GB Expiration-period: 30 days

Disk usage:
detailed: Logs: 137103 MB, Current Retention: 14 days
summary: Logs: 22017 MB, Current Retention: 27 days
infra_audit: Logs: 1403 MB, Current Retention: 21 days
platform: Logs: 0 MB, Current Retention: 0 days
external: Logs: 0 MB, Current Retention: 0 days

Slot:2
Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
summary: 30.00%, 141 GB Expiration-period: 30 days
infra_audit: 5.00%, 24 GB Expiration-period: 30 days
platform: 0.10%, 0 GB Expiration-period: 30 days
external: 0.10%, 0 GB Expiration-period: 30 days

Disk usage:
detailed: Logs: 137118 MB, Current Retention: 14 days
summary: Logs: 21723 MB, Current Retention: 27 days
infra_audit: Logs: 1401 MB, Current Retention: 21 days
platform: Logs: 0 MB, Current Retention: 0 days
external: Logs: 0 MB, Current Retention: 0 days

Space reserved for cores: 0MB

Re: log retention days

rxie — Mon, 02 May 2022 09:57:58 GMT

Hey there,

Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
...

Disk usage:
detailed: Logs: 137161 MB, Current Retention: 14 days

Here "Expiration-period: 30 days" means the Max Day set fort the specific kind of log ”detailed logs“ is 30 days, if a detailed log is older than 30 days, Panorama deletes it. If you don't set a "Max Day", then Panorama only deletes the old logs when the disk is full and new logs must be written. “Current Retention: 14 days” means the oldest detailed logs on the disk is 14 days old.

If the device is working fine, wait for a few moredays and you should be able to see "Expiration-period: 30 days" and also “Current Retention: 30 days”

Re: log retention days

JeffKim — Mon, 02 May 2022 15:18:20 GMT

Thanks Rxie ,

In case of mine , I have never seen over than 16 days .

Disk usage:
detailed: Logs: 136903 MB, Current Retention: 14 days
summary: Logs: 22788 MB, Current Retention: 25 days
infra_audit: Logs: 1488 MB, Current Retention: 1 days
platform: Logs: 0 MB, Current Retention: 0 days
external: Logs: 0 MB, Current Retention: 0 days

Space reserved for cores: 0MB

/dev/sdb1 1.7T 1.1T 560G 67% /opt/panlogs/ld1
/dev/sdd1 1.7T 903G 749G 55% /opt/panlogs/ld3
/dev/sdc1 1.7T 904G 748G 55% /opt/panlogs/ld2

Re: log retention days

rxie — Tue, 03 May 2022 01:25:10 GMT

Then seems the device is not working as expected, maybe you can open a case to PA support.

Re: log retention days

JeffKim — Tue, 03 May 2022 01:52:26 GMT

I had submitted case , but they couldn't give me an answer.

Re: log retention days

JimMcGrady — Thu, 11 Aug 2022 08:44:51 GMT

I have the same issue. Panorama 10.1.5 accepting logs from a number of gateways (most being 9.1.13). Threat log allocation, for example, is 64GB. Expiration Period is 90 days. However the logdb-usage command lists 'Current Retention' as 12 days. 64GB should be enough for millions of log entries allowing for at least 90 days. If i export the entire 12 days of threat logs, that is only 80,000 entries.

The PA tech i spoke with suspects the allocated space is clogged up with indexes rather than with actual logs. However he is not yet sure how to check.

Re: log retention days

EdmarFrancis — Fri, 21 Mar 2025 07:13:07 GMT

@JeffKim @JimMcGrady I hope you are doing well. Any chance that you resolved this concern on the log retention?