STIX and TAXII support

[ScottyAU]

Hi all,

Anyone used minemeld with STIX and TAXII?  While we pretty familure with STIX/TAXII - only just booted minemeld for the first time.

Cheers,

Scotty

[lmori]

Hi Scotty,

yes, there are some MineMeld instances out there retrieving indicators via STIX/TAXII, from TIPs mainly.

There are a couple of prototypes for hailataxii feeds in the prototype library you can check as examples.

Let me know if you need some help in setting up the TAXII Miner.

[ScottyAU]

Thanks for the quick response lmori.

I'll have a look at setting up a miner and see how i get on.

We do require both username/password as well as a client certificate for our taxii server - any idea if that is supported?

Cheers!

Scotty

[lmori]

Hi Scotty,

usernamd and password are supported, client certificate not yet. Is it mandatory for your TAXII server ?

Thanks,

luigi

[ScottyAU]

It is required yeah.

Is this all python under the covers?  In which case if its using the python taxii libs its supported - so might be an easy fix.

If its something else im happy to take a look.

Scotty

[lmori]

Hi Scotty,

under the cover is mostly python, and yes the lib already supports it. It is just that the option is not exposed via config and it is easy an easy fix.

I have created an ER to track this, minemeld-core ER#13

[ScottyAU]

Wow that was quick! 

Should i still have a look or just wait for the ER?  (no idea what your backlog/process is like).

Cheers,

Scotty

[lmori]

Hi Scotty,

exposing the option via prototype is easy, it will be a bit more complex exposing the option via the WebUI.

If you are ok in logging in to the VM to upload the cert, this will land in the next minor release. Otherwise, if you need the WebUI immiediately, you will have to wait a bit more.

But if you want to look at the code yourself, you are welcome 🙂 Just check the code in /opt/minemeld/engine/current/lib/python2.7/site-packages/minemeld/ft/taxii.py

When MineMeld will be made Open Source this will be way easier 🙂

[ScottyAU]

Thanks so much mate - and sorry for the delay in response.

I'll configure a new miner off the halitaxii prototype and let you know how i go.

If i create a new prototype - is that local - or does it get pushed back up somwhere for others to see?

[lmori]

Hi Scotty,

1) nothing is shared automatically by MineMeld

2) if you create a new prototype using the NEW button, it will be saved locally in a separate directory. This way it won't be overridden at the next update.

Please, let me know if you need more details about the TAXII prototype.

Luigi

[MGBerkowitz@wellington.com]

This would be very helpful for those of us in the Financial Industry who want to pull in the FS-ISAC feed.

[lmori]

Hi MGBerkowitz,

I will work on it. We already have customers collecting FS-ISAC indicators in Soltra Edge and then using MineMeld to enforce active indicators from Soltra Edge on our NGFW platforms. This way they can conduct manual analysis of indicators on Soltra Edge and use MineMeld to select the active indicators.

I will keep you posted on the progress of this feature.

Thanks,

luigi

[ScottyAU]

Ok so I hacked in certificate support this afternoon.  This is just that - a hack (it's not configureable and uses the same cert for any taxii collection).

Once the changes were made i cloned the existing halitaxii prototype and created one with the relevant URL, user creds and collection name.  Worked first go!

Does anyone have a list of what STIX vocab is supported by minemeld?  It pulled a chunk of data in - but only showed a very small subset of indicators as a result.  Guessing it doesn't understand everything we publish.  Would be great to get a list so we know what is and isnt supported (and possibly suggest some additions) 🙂

/opt/minemeld/engine/current/lib/python2.7/site-packages/minemeld/ft/taxii.py

def configure(self):
        super(TaxiiClient, self).configure()

        self.discovery_service = self.config.get('discovery_service', None)
        self.username = self.config.get('username', None)
        self.password = self.config.get('password', None)
++        self.key_file = '/opt/certs/browsc-key.pem'
++        self.cert_file = '/opt/certs/browsc-cert.pem'
        self.collection = self.config.get('collection', None)
        self.prefix = self.config.get('prefix', self.name)
        self.ca_file = self.config.get('ca_file', None)
        self.confidence_map = self.config.get('confidence_map', {
            'low': 40,
            'medium': 60,
            'high': 80
        })

    def _build_taxii_client(self):
        result = libtaxii.clients.HttpClient()

        up = urlparse.urlparse(self.discovery_service)

        if up.scheme == 'https':
            result.set_use_https(True)

        if self.username and self.password:
++            result.set_auth_type(libtaxii.clients.HttpClient.AUTH_CERT_BASIC)
--            result.set_auth_type(libtaxii.clients.HttpClient.AUTH_BASIC)
            result.set_auth_credentials({
                'username': self.username,
++                'password': self.password,
--                'password': self.password
++                'key_file': self.key_file,
++                'cert_file': self.cert_file
            })

        if self.ca_file is not None:
            result.set_verify_server(
                verify_server=True,
                ca_file=self.ca_file
            )

        return result

[lmori]

Hi ScottyAU,

that's great ! thanks for testing this. I will include something  similar in the next release.

Currently the TAXII Miner supports indicators with observables of type DomainNameObjectType, AddressObjectType, URIObjectType. It can easily be extened to support additional types.

What type of indicators are you receiving via TAXII ?

Thanks,

luigi

[ScottyAU]

Hi Luigi,

This is STIX packages that we (CERT Australia) produce currently and push out to partners.  We're looking at minemeld in the event we have any partner companies that want to use it to talk to us.

The STIX elements we use are:

* Package

* Indicator

* CourseOfAction

* TTP

* KillChain / KillChainPhase

Our Indicators typically contain one or more Cybox Observables each of which describes a Cybox Object. Our STIX packages will potentially include the following Cybox Object types:

* Address

* DomainName

* EmailAddress

* EmailMessage

* File

* HTTPSession

* SocketAddress

* URI

* WinRegistryKey

Cheers,

Scotty