Output limit?

Announcements

ATTENTION Customers, All Partners and Employees: The Customer Support Portal (CSP) will be undergoing maintenance and unavailable on Saturday, November 7, 2020, from 11 am to 11 pm PST. Please read our blog for more information.

Reply
Highlighted
L4 Transporter

Output limit?

 

Hi,


I run minemeld (standalone) in a virtual machine with 2 cpu, 6GB RAM and 40GB hd. My config has 63 miners (mainly youtube miners and ransomware trackers), 13 aggregators and 30 output nodes. The miners start the job, but when it reach the band "85k-95k indicators", the Minemeld stops mining. The miners get the status "started" and few of them "stopped". It doesn't restart the service each x sec, it just stop mining. I know it stops mining, because one of the Youtube channels has more than 30k videos and the miner only gets few URLs, the same with the ransonware IP trackers.

 

I noticed that my Minemeld hardly aged-out or removes indicators. For example, in a typical dashboard showed in many articles, the monitor presents the number of aged-out or removed indicators as a parable (half sine). In my case, figure bellow, I have flat lines. I thought it could be something related with NTP leading Minemeld to be to out of resources with so many indicators, but the time configuration in my server is perfect.

 

Finally, some time later (there is not a specific interval) the dashbord shows 0 indicators.

 

Could someone give me any tip, advice, help?

 

Thanks in advanced.

 

Highlighted
L7 Applicator

Hi @danilo.souza,

flat lines could be normal. But please could you add the minemeld-engine.log file to the thread? You can download it from System > Engine > Logs.

Please, check it before posting that it does not contain confidential information - especially credentials to access feeds.

 

Thanks,

luigi

Highlighted
L4 Transporter

Hi Luigi,

thank you for the fast reply. Just to be sure, there is no way to send the log just to you, right? I have to add the file here in the forum, right?

Thank you one more time.

L7 Applicator

Hi @danilo.souza,

sure, please send them to lmori@paloaltonetworks.com

 

Thanks,

luigi

Highlighted
L7 Applicator

Hi @danilo.souza,

I checked your logs and it seems a rabbitmq malfunction. Which distribution are you using? How much memory do you have on that instance?

 

luigi

Highlighted
L4 Transporter

Hi Luigi,

I am using the version 0.9.44 for CentOS. That is what I get from the engine log:

 

/opt/minemeld/log/minemeld-engine.log.6:2018-04-15T17:07:19 (2404)launcher.main INFO: Starting mm-run.py version 0.9.44.post1

 

What you mean by how much in memory in that instance? When you refers to rabbitmq, is it a bad news?

 

Thank you again.

Highlighted
L7 Applicator

Highlighted
L4 Transporter

Hi Luigi,

it is disabled (image attached).

Best regards.

Highlighted
L4 Transporter

Hi Luigi,


Is there any other information I can provide to help identifying the problem?

 

Thank you again

Highlighted
L4 Transporter

Hi,

is there anybody else with a similar case that could help in this case? It is really important.

Thanks.

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!