Ubuntu 18.04 install errors

[bsmith2]

Hi There, 

Running into issues trying to install on Ubuntu 18.04

During the install process at the end I get this message:

TASK [minemeld : requirements] ******************************************************************************************************************************************************************
fatal: [127.0.0.1]: FAILED! => {"changed": false, "cmd": "/opt/minemeld/engine/current/bin/pip2 freeze", "msg": "\n:stderr: Traceback (most recent call last):\n File \"/opt/minemeld/engine/current/bin/pip2\", line 6, in <module>\n from pip._internal import main\n File \"/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pip/_internal/__init__.py\", line 5, in <module>\n import logging\n File \"/usr/lib/python2.7/logging/__init__.py\", line 26, in <module>\n import sys, os, time, cStringIO, traceback, warnings, weakref, collections\n File \"/usr/lib/python2.7/weakref.py\", line 14, in <module>\n from _weakref import (\nImportError: cannot import name _remove_dead_weakref\n"}
to retry, use: --limit @/home/timpo/minemeld-ansible/local.retry

And this is the output of journalctl -xe:

-- Unit minemeld.service has begun starting up.
Apr 10 12:40:43 ise-miner mkdir[16946]: /bin/mkdir: cannot create directory ‘/var/run/minemeld’: File exists
Apr 10 12:40:44 ise-miner supervisord[16948]: Traceback (most recent call last):
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/current/bin/supervisord", line 6, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: from supervisor.supervisord import main
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/supervisord.py", line 41, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: from supervisor.options import ServerOptions
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/options.py", line 15, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: import pkg_resources
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 84, in <module>
Apr 10 12:40:44 ise-miner supervisord[16948]: __import__('pkg_resources.extern.packaging.requirements')
Apr 10 12:40:44 ise-miner supervisord[16948]: File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/extern/__init__.py", line 61, in load_module
Apr 10 12:40:44 ise-miner supervisord[16948]: "distribution.".format(**locals())
Apr 10 12:40:44 ise-miner supervisord[16948]: ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the p
Apr 10 12:40:44 ise-miner systemd[1]: minemeld.service: Control process exited, code=exited status=1
Apr 10 12:40:44 ise-miner systemd[1]: minemeld.service: Failed with result 'exit-code'.
Apr 10 12:40:44 ise-miner sudo[16924]: pam_unix(sudo:session): session closed for user root
Apr 10 12:40:44 ise-miner systemd[1]: Failed to start Process Monitoring and Control Daemon.

And here's the output of sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf status:

Traceback (most recent call last):
File "/opt/minemeld/engine/current/bin/supervisorctl", line 6, in <module>
from supervisor.supervisorctl import main
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/supervisorctl.py", line 36, in <module>
from supervisor.options import ClientOptions
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/supervisor/options.py", line 15, in <module>
import pkg_resources
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 84, in <module>
__import__('pkg_resources.extern.packaging.requirements')
File "/opt/minemeld/engine/0.9.60/local/lib/python2.7/site-packages/pkg_resources/extern/__init__.py", line 61, in load_module
"distribution.".format(**locals())
ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.

Any ideas?

[Carlos_Gomes]

Thank you @Paul_Stinson after doing all of this work now the engine just stops I'm still trying to see what is causing it. If it worked for you on Docker which I never checked if there was a guide for this, is there one that you followed? My installation after all the small fixes works but once I added some alien vault 3red party vendor threat feeds it broke. I'm certain removing the taxi client feeds will allow it the engine to run stable but if I cannot get the lists from those feeds what's the point right? And I agree with you it would be good to hear from the dev's on this I even posted questions on github too no answers... it's a pity there aren't so many contributing to what I think is a good product. I too am no developer or even have any python background.

[Paul_Stinson]

No Problems @Carlos_Gomes. I ran the docker instructions from the following URI: https://live.paloaltonetworks.com/t5/MineMeld-Articles/Running-MineMeld-using-Docker/ta-p/289062 Installed it on Centos 7.7 no problems at all. Haven't used the particular miner that your using so not sure if same issue is still there for that particular agent. Would love to see Application updated to support Python3 though as who knows what kind of security bugs might eventual turn up in the container itself if still using all old python packages in the container. Will keep an eye on developments and might move back to Ansible version of Minemeld later, however docker version working pretty good at this stage. I'll just export config and import on Ansible Version once its determined as no longer broken or at least trial it again once we know Python 3 work has been done.

[Carlos_Gomes]

Thank you @Paul_Stinson this is helpful, i will try to see if with the docker version minemeld core works and is stable. Minemeld-ansible version stops working for me after 24 hours or so and i then need to re-run the playbook so it is not sustainable but it didn't need many of those fixes we had to do to minemeld core version which is strange in itself. However i don't want to have to run the playbook every 24 hours, or have to write some sort of script that monitors when its broken to then run the playbook. And like you i agree too, python2.7 being deprecated and this being a sec tool it should be running in the latest versions for better support and security. Hopefully the minemeld developers have been quiet because they are all working away for the community in building a better version and more secure. Time will tell... thanks again.

[DanieleDenisi]

Hello,

I solved the  error below  :

" ImportError: librrd.so.4: cannot open shared object file: No such file or directory"    ;

perfoming the following steps :

• systemctl stop minemeld
• updatedb                     [ update database for locate command ]
• locate -i librrd.so        [ look for available librrd.so libraries . Note . if should find /usr/lib/x86_64-linux-gnu/librrd.so.8  ]
• cd /usr/lib/x86_64-linux-gnu/
• ln -s librrd.so.8 librrd.so.4
• systemctl start minemeld

After this steps , the login fase with default [admin/minemeld  ] credenctial works fine.

Ciao,

Daniele

[Carlos_Gomes]

@Paul_Stinson so i tested the docker version and the same issues out of the box apply when connecting a miner to Alienvault... so annoying this whole thing. only minemeld-ansible release works with it but it doesnt stay up for longer than 24hours. This product is so buggy for me. i wish it just worked.

[Carlos_Gomes]

These are the errors i get. I know on minemeld core i fix it by upgrading the libtaxii to latest version but this is in docker so no idea how to do it here.

2020-02-10T17:38:23 (153)basepoller._polling_loop INFO: Polling AlienVault_Any_Miner
2020-02-10T17:38:23 (153)basepoller._poll ERROR: Exception in polling loop for AlienVault_Any_Miner: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)>
Traceback (most recent call last):
File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 737, in _poll
performed = self._polling_loop()
File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 584, in _polling_loop
iterator = self._build_iterator(now)
File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/taxii.py", line 1131, in _build_iterator
self._discover_services(tc)
File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/taxii.py", line 292, in _discover_services
resp = self._call_taxii_service(self.discovery_service, tc, request)
File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/taxii.py", line 282, in _call_taxii_service
port=port
File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/libtaxii/clients.py", line 337, in call_taxii_service2
response = urllib.request.urlopen(req)
File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/libtaxii/clients.py", line 363, in https_open
return self.do_open(self.get_connection, req)
File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
URLError: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)>

[Carlos_Gomes]

another update i was able to follow my own notes to update the libtaxii within the docker container, i would think that if the container stops running or the linux vm restarts the changes i made will need to be redone. i have yet to test this. 

[Brandon_Hess]

This is still working as of June 2021 on Ubuntu 18.04.5 - thanks so much!