another update i was able to follow my own notes to update the libtaxii within the docker container, i would think that if the container stops running or the linux vm restarts the changes i made will need to be redone. i have yet to test this.  ... View more

These are the errors i get. I know on minemeld core i fix it by upgrading the libtaxii to latest version but this is in docker so no idea how to do it here.   2020-02-10T17:38:23 (153)basepoller._polling_loop INFO: Polling AlienVault_Any_Miner 2020-02-10T17:38:23 (153)basepoller._poll ERROR: Exception in polling loop for AlienVault_Any_Miner: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)> Traceback (most recent call last): File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 737, in _poll performed = self._polling_loop() File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/basepoller.py", line 584, in _polling_loop iterator = self._build_iterator(now) File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/taxii.py", line 1131, in _build_iterator self._discover_services(tc) File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/taxii.py", line 292, in _discover_services resp = self._call_taxii_service(self.discovery_service, tc, request) File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/minemeld/ft/taxii.py", line 282, in _call_taxii_service port=port File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/libtaxii/clients.py", line 337, in call_taxii_service2 response = urllib.request.urlopen(req) File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 429, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 447, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain result = func(*args) File "/opt/minemeld/engine/0.9.66/local/lib/python2.7/site-packages/libtaxii/clients.py", line 363, in https_open return self.do_open(self.get_connection, req) File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open raise URLError(err) URLError: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)> ... View more

@Paul_Stinson so i tested the docker version and the same issues out of the box apply when connecting a miner to Alienvault... so annoying this whole thing. only minemeld-ansible release works with it but it doesnt stay up for longer than 24hours. This product is so buggy for me. i wish it just worked. ... View more

Thank you @Paul_Stinson this is helpful, i will try to see if with the docker version minemeld core works and is stable. Minemeld-ansible version stops working for me after 24 hours or so and i then need to re-run the playbook so it is not sustainable but it didn't need many of those fixes we had to do to minemeld core version which is strange in itself. However i don't want to have to run the playbook every 24 hours, or have to write some sort of script that monitors when its broken to then run the playbook. And like you i agree too, python2.7 being deprecated and this being a sec tool it should be running in the latest versions for better support and security. Hopefully the minemeld developers have been quiet because they are all working away for the community in building a better version and more secure. Time will tell... thanks again. ... View more

Thank you @Paul_Stinson after doing all of this work now the engine just stops I'm still trying to see what is causing it. If it worked for you on Docker which I never checked if there was a guide for this, is there one that you followed? My installation after all the small fixes works but once I added some alien vault 3red party vendor threat feeds it broke. I'm certain removing the taxi client feeds will allow it the engine to run stable but if I cannot get the lists from those feeds what's the point right? And I agree with you it would be good to hear from the dev's on this I even posted questions on github too no answers... it's a pity there aren't so many contributing to what I think is a good product. I too am no developer or even have any python background. ... View more

and a simple full reboot fixed this issue. Finally a working product. Too many changes needed to get minemeld-core working in a stable way. Would be great to have the guys who developed this to move to a supported version of python. Either way being that its an open source product and everyone else has their jobs one cannot ask for much more. hopefully i was able to contribute to the product with my findings. ... View more

**bleep** just as i thought i had this fixed no classes now load...  COMMIT FAILED: Class minemeld.ft.ipop.AggregateIPv4FT in IP_Aggregator not safe to load   any class is failing. ... View more

and finally got this working.... used pycharm to figure out where the errors were. Install newer version of libtaxii by doing: pip install libtaxii --force here is the log of that install: root@:/opt/minemeld/engine/current/local# pip install libtaxii --force DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Collecting libtaxii Downloading libtaxii-1.1.115-py2.py3-none-any.whl (130 kB) |████████████████████████████████| 130 kB 4.6 MB/s Collecting python-dateutil>=1.4.1 Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB) |████████████████████████████████| 227 kB 8.2 MB/s Collecting six>=1.9.0 Downloading six-1.14.0-py2.py3-none-any.whl (10 kB) Collecting lxml>=2.2.3 Downloading lxml-4.5.0-cp27-cp27mu-manylinux1_x86_64.whl (5.7 MB) |████████████████████████████████| 5.7 MB 9.0 MB/s ERROR: minemeld-core 0.9.66 has requirement libtaxii==1.1.107, but you'll have libtaxii 1.1.115 which is incompatible. ERROR: minemeld-core 0.9.66 has requirement lxml==4.1.0, but you'll have lxml 4.5.0 which is incompatible. ERROR: minemeld-core 0.9.66 has requirement six==1.11.0, but you'll have six 1.14.0 which is incompatible.   you need to update the following file /opt/minemeld/engine/current/lib/python2.7/site-packages/minemeld_core-0.9.66.dist-info/metadata   change libtaxii version to whatever the newest is, in my case it was 1.1.115, then lxml to 4.5.0 and six to 1.14.0 so it looks like: Requires-Dist: six (==1.14.0) Requires-Dist: lxml (==4.5.0) Requires-Dist: libtaxii (==1.1.115)   then restart minemeld.         ... View more

Just to add one more thing, Pulling data from taxiclient with AlienVault i get an sslv3 handshake failure. I did try to update gevent and greenlet same ways we did the rrdtool and ujson to no avail. it seems that MineMeld creators need to update this to a later version of python also since 2.7 is old and deprecated i believe.   ... View more

Yes yes yes i got it working too thank you @EdwardMarshall for your input. Now i was also having issues with the rrdtool so i followed the same method as you sent but for the rrdtool. sudo apt install python-rrdtool Followed by cd /opt/minemeld/engine/current/local/lib/python2.7/site-packages Then mv rrdtool.so rrdtool.so.old Finally ln -s /usr/lib/python2.7/dist-packages/rrdtool.x86_64-linux-gnu.so rrdtool.so   Restart Minemeld sudo -u minemeld /opt/minemeld/engine/current/bin/supervisorctl -c /opt/minemeld/local/supervisor/config/supervisord.conf start   If you have issues with the minemeld web or minemeld engine starting and you check the logs and see the errors on flask.ext.login being depreciated use flask_login. find and replace flask.ext.login with flask_login on the aaa.py file under /opt/minemeld/engine/current/lib/python2.7/site-packages/minemeld/flask   good luck Carlos ... View more

so odd @a.jones i have tried this twice, i have rebuilt the server too everytime there are issues. aaa.py reports issues with the flask being deprecated and the web service cannot start, change that to flask_login and then it does but the engine never starts as it has issues with the rrdtool. I have been able to fix that since by installing it on the virtual env but out of the box this minemeld at least for me doesn't run properly. When i do get it working i still have issues connecting to an Alienvault OTX pulse of mine. Complains with sslv3 handshake errors. other sources work fine. Support for this obviously open source so i have to rely on this community and that of github to help. ... View more

@a.jones correct i do too but when i try to login it fails. Can you login and is all working as expected? ... View more

Hi Adrian, i know you said you got a login but were you actually able to login? the minemeld-web service is fatal and doesnt start. I noticed in the logs about flask_login so i changed as recommend in the logs and still fails. /opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/minemeld/flask/aaa.py:21: ExtDeprecationWarning: Importing flask.ext.login is deprecated, use flask_login instead. import flask.ext.login So after changing the flask_login and saving the aaa.py file it then no longer errors out but fails next step on loading the rrdtool.   file "/opt/minemeld/engine/0.9.64/local/lib/python2.7/site-packages/minemeld/flask/metricsapi.py", line 19, in <module> import rrdtool ImportError: librrd.so.4: cannot open shared object file: No such file or directory   so still doesn't work. ... View more

Thank you @a.jones i will try as you suggested to see if that works. ... View more

@bsmith2 how did you create the sym link? I cannot seem to get my head around this. ... View more