Ubuntu 18.04 install errors

Reply
L1 Bithead

Ubuntu 18.04 install errors

Hi There, 

 

Running into issues trying to install on Ubuntu 18.04

 

During the install process at the end I get this message:

 

TASK [minemeld : requirements] ******************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "cmd": "/opt/minemeld/engine/current/bin/pip2 freeze", "msg": "\n:stderr: Traceback (most recent call last):\n File \"/opt/minemeld/engine/current/bin/pip2\", line 6, in <module>\n from pip._internal import main\n File \"/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pip/_internal/__init__.py\", line 5, in <module>\n import logging\n File \"/usr/lib/python2.7/logging/__init__.py\", line 26, in <module>\n import sys, os, time, cStringIO, traceback, warnings, weakref, collections\n File \"/usr/lib/python2.7/weakref.py\", line 14, in <module>\n from _weakref import (\nImportError: cannot import name _remove_dead_weakref\n"}
to retry, use: --limit @/home/timpo/minemeld-ansible/local.retry

 

And this is the output of journalctl -xe:

 

-- Unit minemeld.service has begun starting up.
Apr 10 12:40:43 ise-miner mkdir[16946]: /bin/mkdir: cannot create directory ‘/var/run/minemeld’: File exists
Apr 10 12:40:44 ise-miner supervisord[16948]: Traceback (most recent call last):
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/current/bin/supervisord", line 6, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: from supervisor.supervisord import main
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/supervisord.py", line 41, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: from supervisor.options import ServerOptions
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/options.py", line 15, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: import pkg_resources
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 84, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: __import__('pkg_resources.extern.packaging.requirements')
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/extern/__init__.py", line 61, in load_module
Apr 10 12:40:44 ise-miner supervisord[16948]: "distribution.".format(**locals())
Apr 10 12:40:44 ise-miner supervisord[16948]: ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the p
Apr 10 12:40:44 ise-miner systemd[1]: minemeld.service: Control process exited, code=exited status=1
Apr 10 12:40:44 ise-miner systemd[1]: minemeld.service: Failed with result 'exit-code'.
Apr 10 12:40:44 ise-miner sudo[16924]: pam_unix(sudo:session): session closed for user root
Apr 10 12:40:44 ise-miner systemd[1]: Failed to start Process Monitoring and Control Daemon.

 

And here's the output of sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status:

 

Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/supervisorctl", line 6, in <module>
from supervisor.supervisorctl import main
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/supervisorctl.py", line 36, in <module>
from supervisor.options import ClientOptions
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/options.py", line 15, in <module>
import pkg_resources
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 84, in <module>
__import__('pkg_resources.extern.packaging.requirements')
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/extern/__init__.py", line 61, in load_module
"distribution.".format(**locals())
ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.

 

Any ideas?

L1 Bithead

Re: Ubuntu 18.04 install errors

I was able to work past installation errors using a fresh install of Ubuntu, however now there's another issue.

 

[2019-04-10 15:48:52 +0000] [7647] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 517, in spawn_worker
worker.init_process()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process
super(GeventWorker, self).init_process()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process
self.load_wsgi()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 132, in load_wsgi
self.wsgi = self.app.wsgi()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app
__import__(module)
File "/opt/minemeld/engine/core/minemeld/flask/main.py", line 3, in <module>
app = create_app()
File "/opt/minemeld/engine/core/minemeld/flask/__init__.py", line 67, in create_app
from . import metricsapi # noqa
File "/opt/minemeld/engine/core/minemeld/flask/metricsapi.py", line 19, in <module>
import rrdtool
ImportError: librrd.so.4: cannot open shared object file: No such file or directory
[2019-04-10 15:48:52 +0000] [7647] [INFO] Worker exiting (pid: 7647)
[2019-04-10 15:48:52 +0000] [7644] [INFO] Shutting down: Master
[2019-04-10 15:48:52 +0000] [7644] [INFO] Reason: Worker failed to boot.

L7 Applicator

Re: Ubuntu 18.04 install errors

Hi @bsmith2,

which version of Ubuntu 18.04 are you using? Did you follow the instructions before runnig the playbook?

https://github.com/PaloAltoNetworks/minemeld-ansible#howto-on-ubuntu-1804

 

Luigi

L1 Bithead

Re: Ubuntu 18.04 install errors

Hi Luigi,

 

Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-47-generic x86_64)

 

I followed those steps in order, and it seemed to install correctly.

 

timpo@ise-meld-18:~$ sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status

[sudo] password for timpo:

minemeld-engine                  RUNNING   pid 8924, uptime 5:13:04

minemeld-supervisord-listener    RUNNING   pid 7609, uptime 18:02:19

minemeld-traced                  FATAL     Exited too quickly (process log may have details)

minemeld-web                     FATAL     Exited too quickly (process log may have details)

 

Previously minemeld-traced was running but minemeld-web is always fatal.

 

Thanks!

L7 Applicator

Re: Ubuntu 18.04 install errors

@bsmith2 could you reboot the instance? if it doesn't work, please check the /opt/minemeld/log/minemeld-web.log file for errors

 

Thanks,

Luigi

L1 Bithead

Re: Ubuntu 18.04 install errors

Hi Luigi, 

 

Rebooted the instance and nothing starts, different behavior from previously. Here is the repeating sequence from the weblog:

 

[2019-04-11 10:02:09 +0000] [648] [INFO] Worker exiting (pid: 648)

[2019-04-11 10:02:09 +0000] [645] [INFO] Shutting down: Master

[2019-04-11 10:02:09 +0000] [645] [INFO] Reason: Worker failed to boot.

[2019-04-11 10:02:13 +0000] [652] [INFO] Starting gunicorn 19.5.0

[2019-04-11 10:02:13 +0000] [652] [INFO] Listening at: http://127.0.0.1:5000 (652)

[2019-04-11 10:02:13 +0000] [652] [INFO] Using worker: gevent

[2019-04-11 10:02:13 +0000] [655] [INFO] Booting worker with pid: 655

[2019-04-11 10:02:13 +0000] [655] [ERROR] Exception in worker process

Traceback (most recent call last):

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 517, in spawn_worker

    worker.init_process()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process

    super(GeventWorker, self).init_process()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process

    self.load_wsgi()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 132, in load_wsgi

    self.wsgi = self.app.wsgi()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi

    self.callable = self.load()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load

    return self.load_wsgiapp()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp

    return util.import_app(self.app_uri)

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app

    __import__(module)

ImportError: No module named minemeld.flask.main

[2019-04-11 10:02:13 +0000] [655] [INFO] Worker exiting (pid: 655)

[2019-04-11 10:02:13 +0000] [652] [INFO] Shutting down: Master

[2019-04-11 10:02:13 +0000] [652] [INFO] Reason: Worker failed to boot.

 

Thanks!

L1 Bithead

Re: Ubuntu 18.04 install errors

Update - I re-ran the playbook with the version=master switch and I'm getting the same behavior as yesterday, all components start with the exception of web and the weblog now reflects:

 

[2019-04-11 10:32:24 +0000] [4650] [INFO] Starting gunicorn 19.5.0

[2019-04-11 10:32:24 +0000] [4650] [INFO] Listening at: http://127.0.0.1:5000 (4650)

[2019-04-11 10:32:24 +0000] [4650] [INFO] Using worker: gevent

[2019-04-11 10:32:24 +0000] [4653] [INFO] Booting worker with pid: 4653

[2019-04-11 10:32:25 EDT] [4653] [INFO] MMLogger started

/opt/minemeld/engine/core/minemeld/flask/aaa.py:21: ExtDeprecationWarning: Importing flask.ext.login is deprecated, use flask_login instead.

  import flask.ext.login

[2019-04-11 10:32:25 EDT] [4653] [INFO] Config loaded: {'MINEMELD_LOCAL_LIBRARY_PATH': '/opt/minemeld/local/library', 'USERS_DB': 'wsgi.htpasswd', 'MINEMELD_LOCAL_PROTOTYPE_PATH': '/opt/minemeld/local/prototypes', 'SUPERVISOR_URL': 'unix:///var/run/minemeld/minemeld.sock', 'MINEMELD_PROTOTYPE_PATH': '/opt/minemeld/local/prototypes:/opt/minemeld/prototypes/current', 'MINEMELD_PIP_PATH': '/opt/minemeld/engine/current/bin/pip', 'MINEMELD_TRACED_PURGE_PATH': '/opt/minemeld/engine/current/bin/mm-traced-purge', 'API_AUTH_ENABLED': True, 'MINEMELD_LOG_DIRECTORY_PATH': '/opt/minemeld/log', 'MINEMELD_7Z_PATH': '/usr/bin/7z', 'MINEMELD_RESTORE_PATH': '/opt/minemeld/engine/current/bin/mm-restore', 'MINEMELD_LOCAL_PATH': '/opt/minemeld/local', 'MINEMELD_GIT_PATH': '/usr/bin/git', 'SNS_ENABLED': True}

[2019-04-11 10:32:25 EDT] [4653] [INFO] USERS_DB loaded from /opt/minemeld/local/config/api/wsgi.htpasswd

[2019-04-11 10:32:25 EDT] [4653] [INFO] FEEDS_USERS_DB loaded from /opt/minemeld/local/config/api/feeds.htpasswd

[2019-04-11 10:32:25 EDT] [4653] [DEBUG] Instance uuid = ea51fd66d39349079450431b8c259750

[2019-04-11 10:32:25 +0000] [4653] [ERROR] Exception in worker process

Traceback (most recent call last):

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 517, in spawn_worker

    worker.init_process()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 190, in init_process

    super(GeventWorker, self).init_process()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 122, in init_process

    self.load_wsgi()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 132, in load_wsgi

    self.wsgi = self.app.wsgi()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi

    self.callable = self.load()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load

    return self.load_wsgiapp()

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp

    return util.import_app(self.app_uri)

  File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/gunicorn/util.py", line 357, in import_app

    __import__(module)

  File "/opt/minemeld/engine/core/minemeld/flask/main.py", line 3, in <module>

    app = create_app()

  File "/opt/minemeld/engine/core/minemeld/flask/__init__.py", line 67, in create_app

    from . import metricsapi  # noqa

  File "/opt/minemeld/engine/core/minemeld/flask/metricsapi.py", line 19, in <module>

    import rrdtool

ImportError: librrd.so.4: cannot open shared object file: No such file or directory

[2019-04-11 10:32:25 +0000] [4653] [INFO] Worker exiting (pid: 4653)

[2019-04-11 10:32:25 +0000] [4650] [INFO] Shutting down: Master

[2019-04-11 10:32:25 +0000] [4650] [INFO] Reason: Worker failed to boot.

L1 Bithead

Re: Ubuntu 18.04 install errors

Another update, I managed to get it running by installing the rrdtool dev package and then sym linking rrdtool.so.8.0.1 to rrdtool.so.4 and everything seems stable (for now). 

Highlighted
L7 Applicator

Re: Ubuntu 18.04 install errors

That won't work, v4 and v8 have different entrypoints. I am wondering why your install is looking for librrd.4.so that is not shipped with 18.04. Did you install using ansible? or did you have a previous install using the binary packages?

 

Luigi

L0 Member

Re: Ubuntu 18.04 install errors

Was there a fix to this issue?  I'm running into this exact issue on a fresh install of Minemeld on Ubuntu 18.04.  minemeld-web.log shows the following:

 

  import rrdtool

ImportError: librrd.so.4: cannot open shared object file: No such file or directory

[2019-04-11 10:32:25 +0000] [4653] [INFO] Worker exiting (pid: 1748)

[2019-04-11 10:32:25 +0000] [4650] [INFO] Shutting down: Master

[2019-04-11 10:32:25 +0000] [4650] [INFO] Reason: Worker failed to boot.

 

minemeld-web always shows FATAL, Exited too quickly (process log may have details).

 

Any thoughts?  Thanks in advance.

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!