Ubuntu 18.04 install errors

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Ubuntu 18.04 install errors

L1 Bithead

Hi There, 

 

Running into issues trying to install on Ubuntu 18.04

 

During the install process at the end I get this message:

 

TASK [minemeld : requirements] ******************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "cmd": "/opt/minemeld/engine/current/bin/pip2 freeze", "msg": "\n:stderr: Traceback (most recent call last):\n File \"/opt/minemeld/engine/current/bin/pip2\", line 6, in <module>\n from pip._internal import main\n File \"/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pip/_internal/__init__.py\", line 5, in <module>\n import logging\n File \"/usr/lib/python2.7/logging/__init__.py\", line 26, in <module>\n import sys, os, time, cStringIO, traceback, warnings, weakref, collections\n File \"/usr/lib/python2.7/weakref.py\", line 14, in <module>\n from _weakref import (\nImportError: cannot import name _remove_dead_weakref\n"}
to retry, use: --limit @/home/timpo/minemeld-ansible/local.retry

 

And this is the output of journalctl -xe:

 

-- Unit minemeld.service has begun starting up.
Apr 10 12:40:43 ise-miner mkdir[16946]: /bin/mkdir: cannot create directory ‘/var/run/minemeld’: File exists
Apr 10 12:40:44 ise-miner supervisord[16948]: Traceback (most recent call last):
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/current/bin/supervisord", line 6, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: from supervisor.supervisord import main
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/supervisord.py", line 41, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: from supervisor.options import ServerOptions
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/options.py", line 15, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: import pkg_resources
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 84, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: __import__('pkg_resources.extern.packaging.requirements')
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/extern/__init__.py", line 61, in load_module
Apr 10 12:40:44 ise-miner supervisord[16948]: "distribution.".format(**locals())
Apr 10 12:40:44 ise-miner supervisord[16948]: ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the p
Apr 10 12:40:44 ise-miner systemd[1]: minemeld.service: Control process exited, code=exited status=1
Apr 10 12:40:44 ise-miner systemd[1]: minemeld.service: Failed with result 'exit-code'.
Apr 10 12:40:44 ise-miner sudo[16924]: pam_unix(sudo:session): session closed for user root
Apr 10 12:40:44 ise-miner systemd[1]: Failed to start Process Monitoring and Control Daemon.

 

And here's the output of sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status:

 

Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/supervisorctl", line 6, in <module>
from supervisor.supervisorctl import main
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/supervisorctl.py", line 36, in <module>
from supervisor.options import ClientOptions
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/options.py", line 15, in <module>
import pkg_resources
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 84, in <module>
__import__('pkg_resources.extern.packaging.requirements')
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/extern/__init__.py", line 61, in load_module
"distribution.".format(**locals())
ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.

 

Any ideas?

37 REPLIES 37

L1 Bithead

I was able to work past installation errors using a fresh install of Ubuntu, however now there's another issue.

 

[2019-04-10 15:48:52 +0000] [7647] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 517, in spawn_worker
worker.init_process()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process
super(GeventWorker, self).init_process()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process
self.load_wsgi()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 132, in load_wsgi
self.wsgi = self.app.wsgi()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app
__import__(module)
File "/opt/minemeld/engine/core/minemeld/flask/main.py", line 3, in <module>
app = create_app()
File "/opt/minemeld/engine/core/minemeld/flask/__init__.py", line 67, in create_app
from . import metricsapi # noqa
File "/opt/minemeld/engine/core/minemeld/flask/metricsapi.py", line 19, in <module>
import rrdtool
ImportError: librrd.so.4: cannot open shared object file: No such file or directory
[2019-04-10 15:48:52 +0000] [7647] [INFO] Worker exiting (pid: 7647)
[2019-04-10 15:48:52 +0000] [7644] [INFO] Shutting down: Master
[2019-04-10 15:48:52 +0000] [7644] [INFO] Reason: Worker failed to boot.

Hi @bsmith2,

which version of Ubuntu 18.04 are you using? Did you follow the instructions before runnig the playbook?

https://github.com/PaloAltoNetworks/minemeld-ansible#howto-on-ubuntu-1804

 

Luigi

Hi Luigi,

 

Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-47-generic x86_64)

 

I followed those steps in order, and it seemed to install correctly.

 

timpo@ise-meld-18:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status

[sudo] password for timpo:

minemeld-engine                  RUNNING   pid 8924, uptime 5:13:04

minemeld-supervisord-listener    RUNNING   pid 7609, uptime 18:02:19

minemeld-traced                  FATAL     Exited too quickly (process log may have details)

minemeld-web                     FATAL     Exited too quickly (process log may have details)

 

Previously minemeld-traced was running but minemeld-web is always fatal.

 

Thanks!

@bsmith2 could you reboot the instance? if it doesn't work, please check the /opt/minemeld/log/minemeld-web.log file for errors

 

Thanks,

Luigi

Hi Luigi, 

 

Rebooted the instance and nothing starts, different behavior from previously. Here is the repeating sequence from the weblog:

 

[2019-04-11 10:02:09 +0000] [648] [INFO] Worker exiting (pid: 648)

[2019-04-11 10:02:09 +0000] [645] [INFO] Shutting down: Master

[2019-04-11 10:02:09 +0000] [645] [INFO] Reason: Worker failed to boot.

[2019-04-11 10:02:13 +0000] [652] [INFO] Starting gunicorn 19.5.0

[2019-04-11 10:02:13 +0000] [652] [INFO] Listening at: http://127.0.0.1:5000 (652)

[2019-04-11 10:02:13 +0000] [652] [INFO] Using worker: gevent

[2019-04-11 10:02:13 +0000] [655] [INFO] Booting worker with pid: 655

[2019-04-11 10:02:13 +0000] [655] [ERROR] Exception in worker process

Traceback (most recent call last):

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 517, in spawn_worker

    worker.init_process()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process

    super(GeventWorker, self).init_process()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process

    self.load_wsgi()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 132, in load_wsgi

    self.wsgi = self.app.wsgi()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi

    self.callable = self.load()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load

    return self.load_wsgiapp()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp

    return util.import_app(self.app_uri)

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app

    __import__(module)

ImportError: No module named minemeld.flask.main

[2019-04-11 10:02:13 +0000] [655] [INFO] Worker exiting (pid: 655)

[2019-04-11 10:02:13 +0000] [652] [INFO] Shutting down: Master

[2019-04-11 10:02:13 +0000] [652] [INFO] Reason: Worker failed to boot.

 

Thanks!

Update - I re-ran the playbook with the version=master switch and I'm getting the same behavior as yesterday, all components start with the exception of web and the weblog now reflects:

 

[2019-04-11 10:32:24 +0000] [4650] [INFO] Starting gunicorn 19.5.0

[2019-04-11 10:32:24 +0000] [4650] [INFO] Listening at: http://127.0.0.1:5000 (4650)

[2019-04-11 10:32:24 +0000] [4650] [INFO] Using worker: gevent

[2019-04-11 10:32:24 +0000] [4653] [INFO] Booting worker with pid: 4653

[2019-04-11 10:32:25 EDT] [4653] [INFO] MMLogger started

/opt/minemeld/engine/core/minemeld/flask/aaa.py:21: ExtDeprecationWarning: Importing flask.ext.login is deprecated, use flask_login instead.

  import flask.ext.login

[2019-04-11 10:32:25 EDT] [4653] [INFO] Config loaded: {'MINEMELD_LOCAL_LIBRARY_PATH': '/opt/minemeld/local/library', 'USERS_DB': 'wsgi.htpasswd', 'MINEMELD_LOCAL_PROTOTYPE_PATH': '/opt/minemeld/local/prototypes', 'SUPERVISOR_URL': 'unix:///var/run/minemeld/minemeld.sock', 'MINEMELD_PROTOTYPE_PATH': '/opt/minemeld/local/prototypes:/opt/minemeld/prototypes/current', 'MINEMELD_PIP_PATH': '/opt/minemeld/engine/current/bin/pip', 'MINEMELD_TRACED_PURGE_PATH': '/opt/minemeld/engine/current/bin/mm-traced-purge', 'API_AUTH_ENABLED': True, 'MINEMELD_LOG_DIRECTORY_PATH': '/opt/minemeld/log', 'MINEMELD_7Z_PATH': '/usr/bin/7z', 'MINEMELD_RESTORE_PATH': '/opt/minemeld/engine/current/bin/mm-restore', 'MINEMELD_LOCAL_PATH': '/opt/minemeld/local', 'MINEMELD_GIT_PATH': '/usr/bin/git', 'SNS_ENABLED': True}

[2019-04-11 10:32:25 EDT] [4653] [INFO] USERS_DB loaded from /opt/minemeld/local/config/api/wsgi.htpasswd

[2019-04-11 10:32:25 EDT] [4653] [INFO] FEEDS_USERS_DB loaded from /opt/minemeld/local/config/api/feeds.htpasswd

[2019-04-11 10:32:25 EDT] [4653] [DEBUG] Instance uuid = ea51fd66d39349079450431b8c259750

[2019-04-11 10:32:25 +0000] [4653] [ERROR] Exception in worker process

Traceback (most recent call last):

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 517, in spawn_worker

    worker.init_process()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process

    super(GeventWorker, self).init_process()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process

    self.load_wsgi()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 132, in load_wsgi

    self.wsgi = self.app.wsgi()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi

    self.callable = self.load()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load

    return self.load_wsgiapp()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp

    return util.import_app(self.app_uri)

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app

    __import__(module)

  File "/opt/minemeld/engine/core/minemeld/flask/main.py", line 3, in <module>

    app = create_app()

  File "/opt/minemeld/engine/core/minemeld/flask/__init__.py", line 67, in create_app

    from . import metricsapi  # noqa

  File "/opt/minemeld/engine/core/minemeld/flask/metricsapi.py", line 19, in <module>

    import rrdtool

ImportError: librrd.so.4: cannot open shared object file: No such file or directory

[2019-04-11 10:32:25 +0000] [4653] [INFO] Worker exiting (pid: 4653)

[2019-04-11 10:32:25 +0000] [4650] [INFO] Shutting down: Master

[2019-04-11 10:32:25 +0000] [4650] [INFO] Reason: Worker failed to boot.

Another update, I managed to get it running by installing the rrdtool dev package and then sym linking rrdtool.so.8.0.1 to rrdtool.so.4 and everything seems stable (for now). 

That won't work, v4 and v8 have different entrypoints. I am wondering why your install is looking for librrd.4.so that is not shipped with 18.04. Did you install using ansible? or did you have a previous install using the binary packages?

 

Luigi

Was there a fix to this issue?  I'm running into this exact issue on a fresh install of Minemeld on Ubuntu 18.04.  minemeld-web.log shows the following:

 

  import rrdtool

ImportError: librrd.so.4: cannot open shared object file: No such file or directory

[2019-04-11 10:32:25 +0000] [4653] [INFO] Worker exiting (pid: 1748)

[2019-04-11 10:32:25 +0000] [4650] [INFO] Shutting down: Master

[2019-04-11 10:32:25 +0000] [4650] [INFO] Reason: Worker failed to boot.

 

minemeld-web always shows FATAL, Exited too quickly (process log may have details).

 

Any thoughts?  Thanks in advance.

I have the same issues but on Ubuntu 14:04. Good luck finding an answer. 

i too have this issue. Installed Ubuntu 18.04 then followed the minemeld-ansible instructions. it was working. i restarted the server and it stopped working. Minemeld-web Fatal. 

[2019-10-31 09:33:18 +0000] [4698] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 517, in spawn_worker
worker.init_process()
File "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process
super(GeventWorker, self).init_process()
File "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process
self.load_wsgi()
File "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 132, in load_wsgi
self.wsgi = self.app.wsgi()
File "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app
__import__(module)
File "/opt/minemeld/engine/core/minemeld/flask/main.py", line 3, in <module>
app = create_app()
File "/opt/minemeld/engine/core/minemeld/flask/__init__.py", line 67, in create_app
from . import metricsapi # noqa
File "/opt/minemeld/engine/core/minemeld/flask/metricsapi.py", line 19, in <module>
import rrdtool
ImportError: librrd.so.4: cannot open shared object file: No such file or directory

 

what i dont get is that after installation it started and was working fine then rebooted and issue.

@bsmith2 how did you create the sym link? I cannot seem to get my head around this.

I have found if you install on Ubuntu 18.04 Bionic Beaver server edition it works. it took me two installs with reboots to get it working but I get a log in. I think the first install missed some things. Second install, rerun from the beginning as if a fresh install.

 

regards

 

Adrian

Thank you @a.jones i will try as you suggested to see if that works.

  • 28455 Views
  • 37 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!