URL Logs do not appear for more than 3 days

HKatbay — Sun, 02 Jun 2024 06:11:23 GMT

User Activity sharing the URL logs for only the last three days, even after editing the log storage and retention setting for the URL summary.

Re: URL Logs do not appear for more than 3 days

Claw4609 — Mon, 03 Jun 2024 20:34:10 GMT

Hello,

If you run show system logdb-quota what does it show for the retention of the logs? And to confirm are you looking to increase the retention of the URL logs in the monitor tab or URL related items in the ACC?

Re: URL Logs do not appear for more than 3 days

HKatbay — Tue, 04 Jun 2024 06:52:20 GMT

We need that URL in generated user activity report be at least for 1 month.

this is the output of the command:

Quotas:
system: 4.00%, 0.632 GB Expiration-period: 0 days
config: 4.00%, 0.632 GB Expiration-period: 0 days
alarm: 3.00%, 0.474 GB Expiration-period: 0 days
appstat: 4.00%, 0.632 GB Expiration-period: 0 days
hip-reports: 1.50%, 0.237 GB Expiration-period: 0 days
traffic: 27.00%, 4.268 GB Expiration-period: 0 days
threat: 11.00%, 1.739 GB Expiration-period: 0 days
trsum: 3.50%, 0.553 GB Expiration-period: 0 days
hourlytrsum: 1.50%, 0.237 GB Expiration-period: 0 days
dailytrsum: 1.50%, 0.237 GB Expiration-period: 0 days
weeklytrsum: 1.50%, 0.237 GB Expiration-period: 0 days
urlsum: 2.00%, 0.316 GB Expiration-period: 0 days
hourlyurlsum: 1.50%, 0.237 GB Expiration-period: 0 days
dailyurlsum: 1.50%, 0.237 GB Expiration-period: 0 days
weeklyurlsum: 1.50%, 0.237 GB Expiration-period: 0 days
thsum: 2.00%, 0.316 GB Expiration-period: 0 days
hourlythsum: 1.50%, 0.237 GB Expiration-period: 0 days
dailythsum: 1.50%, 0.237 GB Expiration-period: 0 days
weeklythsum: 1.50%, 0.237 GB Expiration-period: 0 days
userid: 1.50%, 0.237 GB Expiration-period: 0 days
iptag: 1.50%, 0.237 GB Expiration-period: 0 days
application-pcaps: 1.50%, 0.237 GB Expiration-period: 0 days
extpcap: 1.50%, 0.237 GB Expiration-period: 0 days
debug-filter-pcaps: 1.50%, 0.237 GB Expiration-period: 0 days
dlp-logs: 1.50%, 0.237 GB Expiration-period: 0 days
hipmatch: 3.00%, 0.474 GB Expiration-period: 0 days
gtp: 2.00%, 0.316 GB Expiration-period: 0 days
gtpsum: 1.50%, 0.237 GB Expiration-period: 0 days
hourlygtpsum: 1.00%, 0.158 GB Expiration-period: 0 days
dailygtpsum: 1.00%, 0.158 GB Expiration-period: 0 days
weeklygtpsum: 1.00%, 0.158 GB Expiration-period: 0 days
auth: 1.50%, 0.237 GB Expiration-period: 0 days
sctp: 0.00%, 0.000 GB Expiration-period: 0 days
sctpsum: 0.00%, 0.000 GB Expiration-period: 0 days
hourlysctpsum: 0.00%, 0.000 GB Expiration-period: 0 days
dailysctpsum: 0.00%, 0.000 GB Expiration-period: 0 days
weeklysctpsum: 0.00%, 0.000 GB Expiration-period: 0 days
decryption: 1.00%, 0.158 GB Expiration-period: 0 days
desum: 1.00%, 0.158 GB Expiration-period: 0 days
hourlydesum: 0.00%, 0.000 GB Expiration-period: 0 days
dailydesum: 0.00%, 0.000 GB Expiration-period: 0 days
weeklydesum: 0.00%, 0.000 GB Expiration-period: 0 days
globalprotect: 1.50%, 0.237 GB Expiration-period: 0 days

Disk usage:
traffic: Logs and Indexes: 3.4G Current Retention: 15 days
threat: Logs and Indexes: 1.4G Current Retention: 8 days
system: Logs and Indexes: 511M Current Retention: 63 days
config: Logs and Indexes: 272M Current Retention: 354 days
alarm: Logs and Indexes: 40K Current Retention: 0 days
trsum: Logs and Indexes: 575M Current Retention: 5 days
hourlytrsum: Logs and Indexes: 235M Current Retention: 1 days
dailytrsum: Logs and Indexes: 253M Current Retention: 6 days
weeklytrsum: Logs and Indexes: 248M Current Retention: 30 days
thsum: Logs and Indexes: 325M Current Retention: 91 days
hourlythsum: Logs and Indexes: 241M Current Retention: 107 days
dailythsum: Logs and Indexes: 241M Current Retention: 110 days
weeklythsum: Logs and Indexes: 189M Current Retention: 345 days
appstatdb: Logs and Indexes: 644M Current Retention: 49 days
userid: Logs and Indexes: 195M Current Retention: 1 days
iptag: Logs and Indexes: 32K Current Retention: 0 days
hipmatch: Logs and Indexes: 40K Current Retention: 0 days
hip-reports: Logs and Indexes: Current Retention: 0 days
extpcap: Logs and Indexes: 8.1M Current Retention: 354 days
urlsum: Logs and Indexes: 333M Current Retention: 5 days
hourlyurlsum: Logs and Indexes: 223M Current Retention: 1 days
dailyurlsum: Logs and Indexes: 236M Current Retention: 6 days
weeklyurlsum: Logs and Indexes: 234M Current Retention: 23 days
gtp: Logs and Indexes: 8.1M Current Retention: 354 days
gtpsum: Logs and Indexes: 12M Current Retention: 354 days
hourlygtpsum: Logs and Indexes: 3.5M Current Retention: 0 days
dailygtpsum: Logs and Indexes: 3.5M Current Retention: 0 days
weeklygtpsum: Logs and Indexes: 272K Current Retention: 0 days
auth: Logs and Indexes: 8.1M Current Retention: 354 days
sctp: Logs and Indexes: 32K Current Retention: 0 days
sctpsum: Logs and Indexes: 3.5M Current Retention: 0 days
hourlysctpsum: Logs and Indexes: 8.0K Current Retention: 0 days
dailysctpsum: Logs and Indexes: 8.0K Current Retention: 0 days
weeklysctpsum: Logs and Indexes: 8.0K Current Retention: 0 days
decryption: Logs and Indexes: 126M Current Retention: 26 days
desum: Logs and Indexes: 164M Current Retention: 36 days
hourlydesum: Logs and Indexes: 8.0K Current Retention: 0 days
dailydesum: Logs and Indexes: 8.0K Current Retention: 0 days
weeklydesum: Logs and Indexes: 8.0K Current Retention: 0 days
globalprotect: Logs and Indexes: 194M Current Retention: 43 days
application: Logs and Indexes: 218M Current Retention: 120 days
filters: Logs and Indexes: 4.0K Current Retention: 0 days
dlp: Logs and Indexes: 4.0K Current Retention: 0 days
hip_report_base: Logs and Indexes: 2.1M Current Retention: N/A
wildfire: Logs and Indexes: 132K Current Retention: N/A

Re: URL Logs do not appear for more than 3 days

reaper — Tue, 04 Jun 2024 11:35:11 GMT

it looks like you're generating so much log you'll only have a few days worth of logs.

you can try to tune your logDB to not store log for topics you don't need, and tune logging so you only log what is important to you, but seeing the volume you currently generate you'll likely not be able to tweak your way to 30 days.

In this case it would be good to consider

- adopting Panorama with centralized log collectors which gets you far more storage capacity

- strata logging service (cortex data lake) purely to receive logs and generate some reports

- export all logs to a SIEM and generate reports there

topic Re: URL Logs do not appear for more than 3 days in Next-Generation Firewall Discussions

URL Logs do not appear for more than 3 days

Re: URL Logs do not appear for more than 3 days

Re: URL Logs do not appear for more than 3 days

Re: URL Logs do not appear for more than 3 days