log retention days

[JeffKim]

Hi we have 2 panorama and it has virtual disks for log-collector.

I have checked log-collector-es-cluster health and it is green.

1 collector-group and 2 log-collectors 

When i run cli

show system logdb-quota at the active panorama , i get result as below

How can I understand expiration-period is 30 days , but I can't see more than 16 days 

Is it disk volume issue ? IMHO , it looks overwrite traffic log older than 16 days

Quotas:
system: 8.00%, 1.072 GB Expiration-period: 7 days
config: 8.00%, 1.072 GB Expiration-period: 7 days
hip-reports: 1.00%, 0.134 GB Expiration-period: 0 days
appstat: 5.00%, 0.670 GB Expiration-period: 0 days

Disk usage:
system: Logs and Indexes: 844.9MB Current Retention: 7 days
config: Logs and Indexes: 28.8MB Current Retention: 7 days
appstatdb: Logs and Indexes: 691.5MB Current Retention: 20 days
hip-reports: Logs and Indexes: 0 Current Retention: 0 days

Slot:0
Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
summary: 30.00%, 141 GB Expiration-period: 30 days
infra_audit: 5.00%, 24 GB Expiration-period: 30 days
platform: 0.10%, 0 GB Expiration-period: 30 days
external: 0.10%, 0 GB Expiration-period: 30 days

Disk usage:
detailed: Logs: 137161 MB, Current Retention: 14 days
summary: Logs: 21456 MB, Current Retention: 27 days
infra_audit: Logs: 1425 MB, Current Retention: 21 days
platform: Logs: 0 MB, Current Retention: 0 days
external: Logs: 0 MB, Current Retention: 0 days

Slot:1
Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
summary: 30.00%, 141 GB Expiration-period: 30 days
infra_audit: 5.00%, 24 GB Expiration-period: 30 days
platform: 0.10%, 0 GB Expiration-period: 30 days
external: 0.10%, 0 GB Expiration-period: 30 days

Disk usage:
detailed: Logs: 137103 MB, Current Retention: 14 days
summary: Logs: 22017 MB, Current Retention: 27 days
infra_audit: Logs: 1403 MB, Current Retention: 21 days
platform: Logs: 0 MB, Current Retention: 0 days
external: Logs: 0 MB, Current Retention: 0 days

Slot:2
Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
summary: 30.00%, 141 GB Expiration-period: 30 days
infra_audit: 5.00%, 24 GB Expiration-period: 30 days
platform: 0.10%, 0 GB Expiration-period: 30 days
external: 0.10%, 0 GB Expiration-period: 30 days

Disk usage:
detailed: Logs: 137118 MB, Current Retention: 14 days
summary: Logs: 21723 MB, Current Retention: 27 days
infra_audit: Logs: 1401 MB, Current Retention: 21 days
platform: Logs: 0 MB, Current Retention: 0 days
external: Logs: 0 MB, Current Retention: 0 days

Space reserved for cores: 0MB

[rxie]

Hey there,

Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
...

Disk usage:
detailed: Logs: 137161 MB, Current Retention: 14 days

Here "Expiration-period: 30 days" means the Max Day set fort the specific kind of log  ”detailed logs“ is 30 days, if a detailed log is older than 30 days, Panorama deletes it. If you don't set a "Max Day", then Panorama only deletes the old logs when the disk is full and new logs must be written. “Current Retention: 14 days” means the oldest detailed logs on the disk is 14 days old.

If the device is working fine, wait for a few moredays and you should be able to see "Expiration-period: 30 days" and also “Current Retention: 30 days”

[JeffKim]

Thanks Rxie ,

In case of mine , I have never seen  over than 16 days .

Slot:2
Quotas:
detailed: 60.00%, 282 GB Expiration-period: 30 days
summary: 30.00%, 141 GB Expiration-period: 30 days
infra_audit: 5.00%, 24 GB Expiration-period: 30 days
platform: 0.10%, 0 GB Expiration-period: 30 days
external: 0.10%, 0 GB Expiration-period: 30 days

Disk usage:
detailed: Logs: 136903 MB, Current Retention: 14 days
summary: Logs: 22788 MB, Current Retention: 25 days
infra_audit: Logs: 1488 MB, Current Retention: 1 days
platform: Logs: 0 MB, Current Retention: 0 days
external: Logs: 0 MB, Current Retention: 0 days

Space reserved for cores: 0MB

/dev/sdb1 1.7T 1.1T 560G 67% /opt/panlogs/ld1
/dev/sdd1 1.7T 903G 749G 55% /opt/panlogs/ld3
/dev/sdc1 1.7T 904G 748G 55% /opt/panlogs/ld2

[rxie]

Then seems the device is not working as expected, maybe you can open a case to PA support.

[JeffKim]

I had submitted case , but they couldn't give me an answer.

[JimMcGrady]

I have the same issue.  Panorama 10.1.5 accepting logs from a number of gateways (most being 9.1.13).  Threat log allocation, for example, is 64GB. Expiration Period is 90 days.  However the logdb-usage command lists 'Current Retention' as 12 days.   64GB should be enough for millions of log entries allowing for at least 90 days.    If i export the entire 12 days of threat logs, that is only 80,000 entries.

The PA tech i spoke with suspects the allocated space is clogged up with indexes rather than with actual logs. However he is not yet sure how to check.