Hello,
I have a problem with my minemeld server. Some days ago I reached 128 nodes (sum of miners, processors and outputs). If I do a commit with 128 nodes or more the minemeld process seems to keep restarting over and over. As soon as I reduce to 127 nodes or less it works again.
In the logs I did not find a lot ...
2018-07-24T16:18:32 (7301)queryprocessor.__init__ INFO: Query 322ca06b-e087-43db-8970-c52bf86efecc -
2018-07-24T16:18:32 (7301)queryprocessor._core_run INFO: Query 322ca06b-e087-43db-8970-c52bf86efecc finished - 100
2018-07-24T16:18:32 (7301)storage.remove_reference WARNING: Attempt to remove non existing reference: 322ca06b-e087-43db-8970-c52bf86efecc - 000000005b4be000
2018-07-24T16:18:32 (7301)storage.remove_reference WARNING: Attempt to remove non existing reference: 322ca06b-e087-43db-8970-c52bf86efecc - 000000005b551a80
2018-07-24T16:18:32 (7301)storage.remove_reference WARNING: Attempt to remove non existing reference: 322ca06b-e087-43db-8970-c52bf86efecc - 000000005b4d3180
2018-07-24T16:18:32 (7301)storage.remove_reference WARNING: Attempt to remove non existing reference: 322ca06b-e087-43db-8970-c52bf86efecc - 000000005b4fd480
Traceback (most recent call last):
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/gevent/hub.py", line 140, in handle
self.handler(*self.args, **self.kwargs)
File "/opt/minemeld/engine/core/minemeld/traced/main.py", line 75, in _sigterm_handler
raise KeyboardInterrupt('Ctrl-C from _sigterm_handler')
KeyboardInterrupt: Ctrl-C from _sigterm_handler
2018-07-24T16:21:01 (7301)writer.stop INFO: Writer - stop called
2018-07-24T16:21:01 (7301)queryprocessor.stop INFO: QueryProcessor - stop called
2018-07-24T16:21:01 (7301)storage.stop INFO: Store - stop called
2018-07-24T16:27:43 (962)main.main INFO: Starting mm-traced version 0.9.44.post1
2018-07-24T16:27:43 (962)main.main INFO: mm-traced arguments: Namespace(config='/opt/minemeld/local/config/traced/traced.yml', verbose=False)
2018-07-24T16:27:43 (962)main.main INFO: mm-traced config: {}
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-traced", line 11, in <module>
load_entry_point('minemeld-core', 'console_scripts', 'mm-traced')()
File "/opt/minemeld/engine/core/minemeld/traced/main.py", line 136, in main
comm.start()
File "/opt/minemeld/engine/core/minemeld/comm/amqp.py", line 595, in start
c = amqp.connection.Connection(**self.amqp_config)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
self.transport = self.Transport(host, connect_timeout, ssl)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
return create_transport(host, connect_timeout, ssl)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
return TCPTransport(host, connect_timeout)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
raise socket.error(last_err)
socket.error: [Errno 111] Connection refused
Unhandled exception in thread started by
sys.excepthook is missing
lost sys.stderr
2018-07-24T16:27:45 (1939)main.main INFO: Starting mm-traced version 0.9.44.post1
2018-07-24T16:27:45 (1939)main.main INFO: mm-traced arguments: Namespace(config='/opt/minemeld/local/config/traced/traced.yml', verbose=False)
2018-07-24T16:27:45 (1939)main.main INFO: mm-traced config: {}
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-traced", line 11, in <module>
load_entry_point('minemeld-core', 'console_scripts', 'mm-traced')()
File "/opt/minemeld/engine/core/minemeld/traced/main.py", line 136, in main
comm.start()
File "/opt/minemeld/engine/core/minemeld/comm/amqp.py", line 595, in start
c = amqp.connection.Connection(**self.amqp_config)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
self.transport = self.Transport(host, connect_timeout, ssl)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
return create_transport(host, connect_timeout, ssl)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
return TCPTransport(host, connect_timeout)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
raise socket.error(last_err)
socket.error: [Errno 111] Connection refused
2018-07-24T16:27:47 (3575)main.main INFO: Starting mm-traced version 0.9.44.post1
2018-07-24T16:27:47 (3575)main.main INFO: mm-traced arguments: Namespace(config='/opt/minemeld/local/config/traced/traced.yml', verbose=False)
2018-07-24T16:27:47 (3575)main.main INFO: mm-traced config: {}
Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/mm-traced", line 11, in <module>
load_entry_point('minemeld-core', 'console_scripts', 'mm-traced')()
File "/opt/minemeld/engine/core/minemeld/traced/main.py", line 136, in main
comm.start()
File "/opt/minemeld/engine/core/minemeld/comm/amqp.py", line 595, in start
c = amqp.connection.Connection(**self.amqp_config)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
self.transport = self.Transport(host, connect_timeout, ssl)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
return create_transport(host, connect_timeout, ssl)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
return TCPTransport(host, connect_timeout)
File "/opt/minemeld/engine/current/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
raise socket.error(last_err)
socket.error: [Errno 111] Connection refused
2018-07-24T16:27:50 (4843)main.main INFO: Starting mm-traced version 0.9.44.post1
2018-07-24T16:27:50 (4843)main.main INFO: mm-traced arguments: Namespace(config='/opt/minemeld/local/config/traced/traced.yml', verbose=False)
2018-07-24T16:27:50 (4843)main.main INFO: mm-traced config: {}
Is this an existing bug or is there at least a solution on how to get more than 128 nodes?
Thanks in advance for your help.
Remo
... View more