We've installed MM on Ubuntu 14.04 and everything starts and seems to work OK initially.
However, after a period of time it seems tro crash. Not really sure how log, but as an example I booted yesterday used if fine for an hour or so, and this morning it had failed.
A typical error (top right in red box) would be ERROR RETRIEVING MINEMELD CONFIG: Internal Server Error. - see screenshot attachment.
If I restart the minemeld service everything starts and all is good again for a period of time. Nothing jumps out in the logs - is there any advice you can give on things to check?
I have same problem, but my minemeld on Ubuntu 14.05 is running syslog miner/analyzer with significant number of logs per second received from firewall. It crashed every day, after about 20-30h. Luigi advised to add CPU, I have now 4x4 cores (4g ram) . It's up and running since 18h , will see...
Thanks - will look into it.
Our deployment is fairly light - 2GB, 2vCPU - but the only processing we're doing over the default config is 2 new IP sources of ~70k addresses, so no in-line syslog processing.
70k addresses are a really low volume for MineMeld. Would you mind sending me the minemeld-engine.log file from /opt/minemeld/log ? My email address is firstname.lastname@example.org
- could you check memory and disk of the instance to see if they are exhausted ?
- are you using one or more taxii data feed output nodes ? those are memory hungry, next release will cut memory usage of taxii data feeds by more than 75%.
I've uploaded a couple of screenshots to show the current setup:-
Resource_Use =>Triggered a reload of the largest IP list, showing the OS level stats (htop) and MM UI reported stats. Probably a little disingenuous as CPU on the OS hits 100% but only for a few seconds (I missed it with the screenshot), and I suspect that the refresh period on the MM UI means it lags a little.
Nodes => Our nodes: we've created 2 new inputs, 1 aggregator and 1 output, plus the default ones. The inputs are based on the minemeld.ft.http.HttpFT prototype
Flows => Connections
Will attach the log to our another message as looks like 3 is max...
the volume of indicators I see from your screenshot should be handled pretty well by MineMeld with those memory and CPU resources. Would you mind uploading also the /opt/minemeld/log/minemeld-web.log file ?
If you prefer you can send it directly to me at lmori@palo...
do you just reboot the instance or do something more ?
Could you run this command before reboot to check which process is using most of the memory ?
$ top -b -n 1 -o %MEM
About the disk, are you erasing files before reboot ? I am asking because it's strange that a reboot alone could free space from disk.
apackard problem should be different, his instance is handling a pretty low volume of indicators.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!
The Live Community thanks you for your participation!