This website uses Cookies. By clicking Accept, you agree to the storing of cookies on your device to enhance your community and translation experience. Read our Privacy Policy.
Click Preferences to customize your cookie settings.
Accept
Reject
Preferences

MineMeld Docker

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements

MineMeld Docker

SabreAce33
L2 Linker

‎07-25-2016 12:50 PM - edited ‎07-26-2016 06:10 AM

I started building out a very simple dev (read: unhardened) docker build for MineMeld here: https://github.com/swannysec/MineMeldDocker

 

Looks like it won't start up correctly inside a container and I think it might be related to the use of UNIX sockets and/or something to do with supervisor.  Anyway, feel free to take a stab at it!  I have to put it aside for a few weeks, but feel free to submit PRs if you find something.

6 REPLIES 6

lmori
L7 Applicator

‎07-25-2016 01:08 PM

Hi SabreAce33,

that's super interesting. Also @nbilal was interested in running MineMeld in a docker. I have forked your repo, will take a look at it.

 

Thanks !

luigi

0 Likes Likes

lmori
L7 Applicator
In response to lmori

‎09-20-2016 10:47 PM

@SabreAce33, @nbilal I have started working on the docker container for MineMeld. Still unstable, do not use it in production.

 

You can pull it from docker hub using:

docker pull jtschichold/minemeld

Git repo available at https://github.com/jtschichold/minemeld-docker

nbilal
L3 Networker

‎10-26-2016 09:21 AM - edited ‎10-26-2016 09:52 AM

 

 When I first run the container, I have to start up the redis-server manually in order to get to the web UI and login. After successful login, I see "ERROR RETRIEVING X: INTERNAL SERVER ERROR" and "ERROR RETRIEVING X: 500" on every page. minemeld-web.log displays the following:

root@40ff97c925f4:/# tail -n 100 /opt/minemeld/log/minemeld-web.log 
Exception on /traced/query [GET]
Traceback (most recent call last):
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
 response = self.full_dispatch_request()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
 rv = self.handle_user_exception(e)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
 reraise(exc_type, exc_value, tb)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
 rv = self.dispatch_request()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
 return self.view_functions[rule.endpoint](**req.view_args)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask_login.py", line 758, in decorated_view
 return func(*args, **kwargs)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/tracedapi.py", line 73, in traced_query
 'query': query
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/__init__.py", line 122, in send_cmd
 self._open_channel()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/__init__.py", line 119, in _open_channel
 self.comm.start()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 545, in start
 amqp.connection.Connection(**self.amqp_config)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
 self.transport = self.Transport(host, connect_timeout, ssl)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
 return create_transport(host, connect_timeout, ssl)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
 return TCPTransport(host, connect_timeout)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
 raise socket.error(last_err)
error: [Errno 111] Connection refused
127.0.0.1 - - [26/Oct/2016:16:13:41 +0000] "GET /traced/query?nl=100&ts=1477498421474&uuid=6ee01212-ede5-4287-9cd9-dd17042ad65f HTTP/1.0" 500 291 "https://localhost:8443/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36"
Loaded user: admin
Loaded user: admin
Exception on /traced/query [GET]
Traceback (most recent call last):
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
 response = self.full_dispatch_request()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
 rv = self.handle_user_exception(e)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
 reraise(exc_type, exc_value, tb)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
 rv = self.dispatch_request()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
 return self.view_functions[rule.endpoint](**req.view_args)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask_login.py", line 758, in decorated_view
 return func(*args, **kwargs)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/tracedapi.py", line 73, in traced_query
 'query': query
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/__init__.py", line 122, in send_cmd
 self._open_channel()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/__init__.py", line 119, in _open_channel
 self.comm.start()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 545, in start
 amqp.connection.Connection(**self.amqp_config)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
 self.transport = self.Transport(host, connect_timeout, ssl)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
 return create_transport(host, connect_timeout, ssl)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
 return TCPTransport(host, connect_timeout)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
 raise socket.error(last_err)
error: [Errno 111] Connection refused
127.0.0.1 - - [26/Oct/2016:16:13:44 +0000] "GET /traced/query?nl=200&ts=1477498424532&uuid=7522a48c-d14a-4265-87af-40bd8f7ea5fc HTTP/1.0" 500 291 "https://localhost:8443/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36"
Loaded user: admin
Loaded user: admin
Exception on /traced/query [GET]
Traceback (most recent call last):
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
 response = self.full_dispatch_request()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
 rv = self.handle_user_exception(e)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
 reraise(exc_type, exc_value, tb)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
 rv = self.dispatch_request()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
 return self.view_functions[rule.endpoint](**req.view_args)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/flask_login.py", line 758, in decorated_view
 return func(*args, **kwargs)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/tracedapi.py", line 73, in traced_query
 'query': query
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/__init__.py", line 122, in send_cmd
 self._open_channel()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/flask/__init__.py", line 119, in _open_channel
 self.comm.start()
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 545, in start
 amqp.connection.Connection(**self.amqp_config)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
 self.transport = self.Transport(host, connect_timeout, ssl)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
 return create_transport(host, connect_timeout, ssl)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
 return TCPTransport(host, connect_timeout)
 File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
 raise socket.error(last_err)
error: [Errno 111] Connection refused
127.0.0.1 - - [26/Oct/2016:16:13:47 +0000] "GET /traced/query?nl=200&ts=1477498427729&uuid=8e85bce3-58ee-4de7-bca0-f16ed5a0a5be HTTP/1.0" 500 291 "https://localhost:8443/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36"
root@40ff97c925f4:/#

And minemeld-engine.log displays the following:

root@40ff97c925f4:/# tail -n 100 /opt/minemeld/log/minemeld-engine.log 
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/chassis.py", line 100, in configure
    self.mgmtbus.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/mgmtbus.py", line 373, in start
    self.comm.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 545, in start
    amqp.connection.Connection(**self.amqp_config)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
    self.transport = self.Transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
    return create_transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused
Process Process-5:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 45, in _run_chassis
    c.configure(fts)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/chassis.py", line 100, in configure
    self.mgmtbus.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/mgmtbus.py", line 373, in start
    self.comm.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 545, in start
    amqp.connection.Connection(**self.amqp_config)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
    self.transport = self.Transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
    return create_transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused
2016-10-26T16:48:46 (14247)launcher._run_chassis ERROR: Exception in chassis main procedure
Traceback (most recent call last):
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 45, in _run_chassis
    c.configure(fts)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/chassis.py", line 100, in configure
    self.mgmtbus.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/mgmtbus.py", line 373, in start
    self.comm.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 545, in start
    amqp.connection.Connection(**self.amqp_config)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
    self.transport = self.Transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
    return create_transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 45, in _run_chassis
    c.configure(fts)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/chassis.py", line 100, in configure
    self.mgmtbus.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/mgmtbus.py", line 373, in start
    self.comm.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 545, in start
    amqp.connection.Connection(**self.amqp_config)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
    self.transport = self.Transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
    return create_transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
error: [Errno 111] Connection refused
Traceback (most recent call last):
  File "/opt/minemeld/engine/current/bin/mm-run", line 11, in <module>
    sys.exit(main())
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 187, in main
    config['nodes'].keys()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/run/launcher.py", line 71, in _start_mgmtbus_master
    mbusmaster.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/mgmtbus.py", line 323, in start
    self.comm.start()
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/minemeld/comm/amqp.py", line 545, in start
    amqp.connection.Connection(**self.amqp_config)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
    self.transport = self.Transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport
    return create_transport(host, connect_timeout, ssl)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport
    return TCPTransport(host, connect_timeout)
  File "/opt/minemeld/engine/0.9.24/local/lib/python2.7/site-packages/amqp/transport.py", line 95, in __init__
    raise socket.error(last_err)
socket.error: [Errno 111] Connection refused
root@40ff97c925f4:/#

 

It does indeed sound like a socket related error, have no idea what that is or how to fix it, but hopefully somebody more qualified will find this info useful. Maybe something broken in RabbitMQ configuration that needs to be adapted to docker setup, as found here and here?

 

0 Likes Likes

lmori
L7 Applicator
In response to nbilal

‎10-27-2016 01:18 AM

Hi @nbilal,

please, could you post your docker logs ? you can grab them using the following command (I am pretty sure you already know this):

docker logs <container>

It seems a problem with RabbitMQ in your environment, could you check:

- if IPv6 is enabled on the docker host ? RabbitMQ release in Ubuntu 14.04 has a bug and you need IPv6 enabled on the docker host even if you are not forwarding it. That's why in the default MineMeld VM IPv6 is enabled but blocked via iptables.

- the /var/log/rabbitmq/startup_err log file ?

 

Thanks,

luigi

0 Likes Likes

nbilal
L3 Networker
In response to lmori

‎11-14-2016 11:55 AM

Just updated my docker-compose this morning o 1.90rc and thought I'd try "docker-compose up" again just for kicks...and everything's working! Did you make any changes to your container on hub.docker.com @lmori?

 

nbilal@e6430 ~/Docker/minemeld $ docker inspect minemeld_minemeld_1 | grep Image
        "Image": "sha256:0284c3d01bed0094a4466b501207cedefca100a969b64f7389c5107674a9d9cd",
            "Image": "jtschichold/minemeld",

 

Keep up the great work!

0 Likes Likes

lmori
L7 Applicator
In response to nbilal

‎11-14-2016 02:13 PM

Yes, when I released 0.9.26 I have added some improvements to the stability of the docker image and the docker-compose file.

 

Glad it worked !

 

0 Likes Likes
  • 17837 Views
  • 6 replies
  • 1 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!

LEGAL NOTICES
RESOURCES
Khoros awards 2022
Copyright 2007 - 2024 - Palo Alto Networks