New stix/taxii miner using cabby

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

New stix/taxii miner using cabby

L2 Linker

I created a new stix/taxii miner for MineMeld, it can be found on github: https://github.com/mr-torgue/mmcabby.

It was created because I encountered severel problems with the default taxii miner and the ng miner. In general mmcabby is more stable because it uses cabby (from eclecticIQ developers of stix/taxii). It also contains support for certificate based authentication.

 

Improvements/remarks/bug notifications are appreciated.

 

8 REPLIES 8

L7 Applicator

Awesome! Are you planning to add UI to it?

Yes, that is one of the things I want to add in a future version. For now everything seems to be working well, so I don't know when I will work on it.

Hi @folmer i tried to get this going as according to your github instructions and doesn't work, i get a lot of errors. For example:

oader._initialize_entry_point_group ERROR: minemeld.ft.local.YamlURLFT not loadable: pytz==2019.3 not compatible with pytz==2015.4, libtaxii==1.1.114 not compatible with libtaxii==1.1.107

 

also the engine now is FATAL and doesnt load properly and i do not see this as an available prototype.

Hello Carlos, did you change the entries for pytz and libtaxii in requirements.txt? Usually requirements.txt contains "pytz==2015.4" and libtaxii"=="1.1.107". However for cabby to work these need to be newer versions. So the requirements have to be changed to "pytz>=2015.4" and libtaxii>="1.1.107". You can probably just restart the service and it should work. 

@folmer Correct. i actually didnt have the folder core under opt/minemeld/engine/ so i created to match your instructions. I created the requirements.txt as per minemeld file in their github and changed the requirements for pytz to pytz==2019.3 and libtaxii to libtaxii==1.1.114, i think below its what you meant to write.

The errors in minemeld.engine log below as example i get that for every prototype.

(26093)loader._initialize_entry_point_group ERROR: minemeld.ft.taxii.TaxiiClient not loadable: pytz==2019.3 not compatible with pytz==2015.4, libtaxii==1.1.114 not compatible with libtaxii==1.1.107

 

I can try this again but havent had much luck leveraging cabby, i was try to do that as with AlienVault OTX i am getting sslv3 handshake failures.

 

(26093)config._load_and_validate_config_from_file ERROR: Invalid config /opt/minemeld/local/config/committed-config.yml: Class minemeld.ft.taxii.TaxiiClient in Cyrebro10_OTX_Pulses not safe to load

 

@folmer i ended up uninstalling minemeld, upgraded to Ubuntu 18.04 and then deployed minemeld-ansible instead or minemeld-core

now working a treat with cabby needed. sslv3 handshake errors gone.

i cannot start minemeld-web service but that is a different issue altogether for another post.

thank you for your reply and help.

L0 Member

Hi @folmer. Have you had any success getting this extension to work with docker MineMeld distribution?

The engine/core directory didn't exist here.

I tried downloading the core git repo to engine/core, modifying the requirements.txt as recommended, but get errors when running "/opt/minemeld/engine/current/bin/python setup.py install".

 

creating build/temp.linux-x86_64-2.7/minemeld/packages/gdns
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DHAVE_NETDB_H= -I/usr/include/python2.7 -c minemeld/packages/gdns/_ares.c -o build/temp.linux-x86_64-2.7/minemeld/packages/gdns/_ares.o
unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

 

Any advice would be greatly appreciated. Thanks.

I solved the dependency issues with an ugly fix. It will work however.

 

I dont know if the developer of minemeld is reading this, but why are minemeld python requirements version specific? For example pytz needs to be version 2015.4, which is pretty old. Also requests needs to be version 2.20 and libtaxii needs to be 1.107. If == could be replaced by >= there would be less dependency issues.

  • 7027 Views
  • 8 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!