Reply
Highlighted
L2 Linker
Posts: 13
Registered: ‎08-15-2014

MineMeld engine:fatal message

I'm getting the below message in my minemeld logs and not sure what is causing it  

 

2018-07-11T00:30:28 (16652)config._destroy_old_nodes INFO: Destroyed nodes: [_ConfigChange(nodename=u'Amazon_IPv4_Agg_General', nodeclass=u'minemeld.ft.ipop.AggregateIPv4FT', change=1, detail={'inputs': ['Amazon_AWS', 'Amazon_CloudFront', 'Amazon_EC2', 'Amazon_S3', 'Amazon_Route53_Agg'], 'config': {'whitelist_prefixes': ['Amazon', 'wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'"], 'name': 'accept IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}), _ConfigChange(nodename=u'compromisedIPs-1531162062986', nodeclass=u'minemeld.ft.http.HttpFT', change=1, detail={'inputs': [], 'config': {'url': 'https://rules.emergingthreats.net/open/suricata/rules/compromised-ips.txt', 'attributes': {'direction': 'inbound', 'type': 'IPv4', 'confidence': 50, 'share_level': 'green'}, 'source_name': 'ET.compromised_ips'}, 'class': 'minemeld.ft.http.HttpFT', 'output': True}), _ConfigChange(nodename=u'wood_IPagg', nodeclass=u'minemeld.ft.ipop.AggregateIPv4FT', change=1, detail={'inputs': ['compromisedIPs-1531162062986', 'blockIPs-1531162050810'], 'config': {'whitelist_prefixes': ['wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'"], 'name': 'accept IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}), _ConfigChange(nodename=u'Amazon_Route53_Agg', nodeclass=u'minemeld.ft.ipop.AggregateIPv4FT', change=1, detail={'inputs': ['Amazon_ROUTE53', 'Amazon_Route53_HealthChecks'], 'config': {'whitelist_prefixes': ['Amazon', 'wl'], 'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ["type == 'IPv4'"], 'name': 'accept IPv4', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.ipop.AggregateIPv4FT', 'output': True}), _ConfigChange(nodename=u'feedMCGreen-1531163955644', nodeclass=u'minemeld.ft.redis.RedisSet', change=1, detail={'inputs': ['wood_IPagg'], 'indicator_types': ['any'], 'node_type': 'output', 'output': False, 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence >= 50', 'confidence < 75', "share_level == 'green'"], 'name': 'accept confidence 50-75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet'}), _ConfigChange(nodename=u'blockIPs-1531162050810', nodeclass=u'minemeld.ft.http.HttpFT', change=1, detail={'inputs': [], 'config': {'url': 'http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt', 'attributes': {'confidence': 50, 'type': 'IPv4', 'share_level': 'green'}, 'source_name': 'ET.block_ips', 'ignore_regex': '^#'}, 'class': 'minemeld.ft.http.HttpFT', 'output': True}), _ConfigChange(nodename=u'wood_output', nodeclass=u'minemeld.ft.redis.RedisSet', change=1, detail={'inputs': ['wood_IPagg'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False}), _ConfigChange(nodename=u'Amazon_Feed', nodeclass=u'minemeld.ft.redis.RedisSet', change=1, detail={'inputs': ['Amazon_IPv4_Agg_General'], 'config': {'infilters': [{'conditions': ["__method == 'withdraw'"], 'name': 'accept withdraws', 'actions': ['accept']}, {'conditions': ['confidence > 75', "share_level == 'green'"], 'name': 'accept confidence > 75 and share level green', 'actions': ['accept']}, {'name': 'drop all', 'actions': ['drop']}]}, 'class': 'minemeld.ft.redis.RedisSet', 'output': False})]
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-run", line 11, in <module>
sys.exit(main())
File "/opt/minemeld/engine/0.9.48.post1/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 218, in main
config = minemeld.run.config.load_config(args.config)
File "/opt/minemeld/engine/0.9.48.post1/local/lib/python2.7/site-packages/minemeld/run/config.py", line 567, in load_config
return _load_config_from_dir(config_path)
File "/opt/minemeld/engine/0.9.48.post1/local/lib/python2.7/site-packages/minemeld/run/config.py", line 417, in _load_config_from_dir
_destroy_old_nodes(cconfig)
File "/opt/minemeld/engine/0.9.48.post1/local/lib/python2.7/site-packages/minemeld/run/config.py", line 357, in _destroy_old_nodes
dpool = multiprocessing.Pool()
File "/usr/lib/python2.7/multiprocessing/__init__.py", line 232, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild)
File "/usr/lib/python2.7/multiprocessing/pool.py", line 138, in __init__
self._setup_queues()
File "/usr/lib/python2.7/multiprocessing/pool.py", line 234, in _setup_queues
self._inqueue = SimpleQueue()
File "/usr/lib/python2.7/multiprocessing/queues.py", line 352, in __init__
self._rlock = Lock()
File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 147, in __init__
SemLock.__init__(self, SEMAPHORE, 1, 1)
File "/usr/lib/python2.7/multiprocessing/synchronize.py", line 75, in __init__
sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 30] Read-only file system

 

I have rebooted the system but that hasn't cleared anything up.  

L0 Member
Posts: 2
Registered: ‎07-11-2018

Re: MineMeld engine:fatal message

 

I work with johnsonto, and can add some context to the error log. 

First, we had two miners aggregating together into an output, and that's been working for some time.

 

Another two-miner to output pair was made, and while the config had a mismatch between confidence levels (such that nothing was being output), the MineMeld server was still working as expected.

 

I changed the prototype of the new output node for a lower confidence, and also created a few more independent miners which connected to a new third output. After this change, the above error reared up, and is preventing our MineMeld engine from running. We're able to commit a new config, but when the engine tries to start up, it continues to crash, always on the same "OSError: Read-only file system". 

 

So far we've been able to change the config, and that leaves me wondering what exactly is read-only. 

L0 Member
Posts: 2
Registered: ‎07-11-2018

Re: MineMeld engine:fatal message

In case anyone else runs into this, we were unable to find a solution and rolled back the VM hosting the server as an alternate solution. 

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!