STIX/TAXII feed not working for otx.alienvault.com

Reply
L2 Linker

STIX/TAXII feed not working for otx.alienvault.com

Hello,

 

I tried to create a STIX/TAXII miner for otx.alienvault.com. I used the default MineMeld taxii client for this(minemeld.ft.taxii.TaxiiClient) and the new client minemeld-taxii-ng(taxiing.Miner).

 

The first client does returns the error ' module object has no attribute 'sslwrap''.
The second returns a 406 client error, which leads me to assume that something is off the accept header.

When trying another server like hailataxii.com the second client(ng) works fine. The first client does not give a SSL error but does not load IOC's.

 

In all cases, when I use cabby it works.

 

So my questions are:

  1. Why does the first Taxii client not work and why the sslwrap error?
  2. Why does the second client return a 406?
  3. Since cabby works pretty well: can it be turned into a node? It would remove the need for a custom STIX client.
  4. Why does the new Taxii client(ng) not use libtaxii anymore? Cabby uses that as well.

Hope you can help me.

 

Best regards,

 

Folmer

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

I figured out a way to get rid of the 'module object has no attribute sslwrap' error. The 'gevent' python package has to be updated.

If MineMeld is already installed:

  1. Go to /<mm>/engine/core
  2. edit requirements.txt, change to gevent==1.1.1, greenlet==0.4.15
  3. /<mm>/engine/current/python setup.py install

Verify that the right version is installed: /<mm>/engine/current/pip install gevent or /<mm>/engine/current/pip freeze. Restart minemeld: "service minemeld restart". The SSL should now be gone.

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

I cannot get MineMeld integrated with otx.alienvault.com using the API key that i was given by my vendor. Basically the vendor gave me their url for their pulses an API key and a username, has anyone been able to integrate minemeld with a specific pulse with otx?

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

The STIX/TAXII client for OTX collects pulses from a user or group. For example the collection user_AlienVault contains all the pulses AlienVault has published. If you want to include other pulses you have several options:

1) poll the user to which the IOC belongs by using collection user_[OTX_username]

2) add the IOC to a group and use collection group_[group_name]

 

The OTX STIX/TAXII implementation is described here: https://otx.alienvault.com/api .

Which vendor are you collecting from?

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

Thank you i got it working with your help. We are picking up lists from a vendor that leverages OTX to create them and distribute them. My only issue now is that i created the miners, then the Aggregators and finally outputs. I see for example URLs from the miner passing through to the aggregator then the aggregator to the output. the line between the aggregator and output shows 200 URLs then the Output icon itself shows 0 indicators, surely shouldnt it show 200?

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

I think it depends on the type of output. The standard outputs (stdlib_feed prototype) work fine. However we also have a CEF output which always displays 0, which might be a bug. What kind of output do you use? I agree that the output should show 200 in your case but I am not sure why it doesn't show it.

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

I have actually tried with a few. I also setup the FSISAC feed and even that also has the same problem.

i have tried to use class minemeld.ft.taxii.DataFeed and minemeld.ft.redis.RedisSet

On the PaloAlto firewalls when i try to ingest the feed from minemeld for ipv4 output, the edl refresh task initially showls EDL(vsys1/"name") downloaded file is not a text file. EDL (vsys1/"name") no valid ips found in list file. Then once job completes it says too many messages. please see job details.

If i look on minemeld and the number of indicators for this output still says 0 yet it shows the aggregator passing through 423 objects. Perhaps i need to use a different class of output?

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

The minemeld.ft.redis.RedisSet output should show the number of IOC's. I have not worked with the taxii output node, so I don't know about that. Do you see the IOC's when you go to the minemeld.ft.redis.RedisSet output node and click on 'FEED BASE URL'? Palo Alto firewalls should be able to use output miners of the Minemeld.ft.redis.RedisSet class.

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

Tried a different config details for the stdlib redis output and i am now  getting this loaded to the PaloALto.

L2 Linker

Re: STIX/TAXII feed not working for otx.alienvault.com

It works with the FSISAC miner feeding to the IP aggregator then feeding to the rediset output. Removed the FSISAC miner so i would only have the other vendor from OTX and i see 36 IOCs passing through the aggregator to the same output but then shows 0. It must have to do with how they miner passes the data to the aggregator... perhaps i need to configure the miner with the same config as the FSISAC one. Going to try that.

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!