Log "Number of hints on disk has exceeded 5000 due to log forward failures."

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Log "Number of hints on disk has exceeded 5000 due to log forward failures."

L4 Transporter

Hi,

 

We are receiving these logs. We would like to know what is causing these logs and how to solve it.

 

hints.JPG

 

Thanks 🙂

13 REPLIES 13

L4 Transporter

any idea?

Hey @BigPalo

 

Check out the below thread, it seems people have resolved the issue by running the command "debug software restart process log-receiver"

 

https://live.paloaltonetworks.com/t5/General-Topics/General-PA-5220/m-p/192473#M57806

 

As for the root cause, are you running Panorama?

 

Cheers,

Luke.

Yes, we are running Panorama

Hey @BigPalo

 

Cheers for confirming that. Did you restart the log receiver service and did it resolve the issue?

 

From what I gather, this problem is caused by the send queue being filled up when attempting to forward logs to Panorama. This can be verified by looking at the netstat output "show netstat" and looking at the "Send Queue" column for a socket open on port 10000.

 

In Panorama, there are a few best practices that we can look at:

 

1. Has a log forwarding preference list been configured? Panorama -> Collector Groups -> Device Log Forwarding

2. Is "enable redundancy across log collectors" checked?

3. Is "Forward to all collectors in the preference list" checked?

 

If options two and three are enabled, without the use of the preference list, then all logs will just be sent to one LC, and this will then be copying the logs to the other LCs anyways - causing a lot of stress. At this point the Panorama will start to throttle logs and this is when you will notice the netstat queues increasing.

 

Cheers,

Luke.



 

In Panorama, there are a few best practices that we can look at:

 

1. Has a log forwarding preference list been configured? Panorama -> Managed Collectors -> Device Log Forwarding



@LukeBullimore- I think that setting is under the Collector Groups, not Managed Collectors

 

Good best practices list - much appreciated!

Hey @JW6224

 

Whoops yeah that was a typo, I'll correct it now. 

 

 

I am still getting this error i ran the command debug restart log receiver

MP

Help the community: Like helpful comments and mark solutions.

I see PA is conected to Panorama and we have dedicated log collectors

 

MP

Help the community: Like helpful comments and mark solutions.

are you still having this issue???

no.

 

restarting the log receiver from the root fixed the issue

MP

Help the community: Like helpful comments and mark solutions.

Hi,

 

What do you exactly mean by restarting the service from the root ?

I'm experiencing the same issue but restarting the service did not resolve the issue on my device ( which is even a standby device )

Do you have any news for this issue? we are still having the problem.

 

Hi,

 

The error dissapeared after restarting the mgmtsrv service and waiting for 8 hours.

We didn't notice the error anymore since then

  • 8115 Views
  • 13 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!