Integrating MineMeld with IBM QRadar

[thanzeeer]

Hi,

I am new to minemeld. I went through the documentation for integrating minemeld with qradar. Succesfully added the TAXII feeds in Qradar.

I couldnt see any values getting populated in reference set defined in Qradar or updates shown in threat intellegence TAXII configuration section in qradar.

Regards

Thanzeer

[lmori]

Hi @thanzeeer,

please could you post a screenshot of your NODES view ?

Thanks,

luigi

[JordDurh]

Hi.

I believe there is an issue with the IP feed within TAXII.

As per documention for setting this up, you create an IP reference set, now this type of reference set can only contain individual host IP's and not ranges, CIDRs etc.

When comparing the normal output lists (I have duplicated my TAXII output into a standard output) I can see this uses IP ranges (or CIDRs if you use ?tr=1), if this is the same as what is contained within the TAXII feed then this will not work within QRadar.

As an example, my TAXII feed has several thousand indicators, however within QRadar it is only processing 4 indicators. I believe what I have mentioned above is the cause.

Can you confirm if this is the case and if there is anyway around it?

[lmori]

Hi @JordDurh,

please note that TAXII DataFeeds work differently from the plain text feeds generated by MineMeld. See this answer for the difference:

https://live.paloaltonetworks.com/twzvq79624/board/message?board.id=MineMeldDiscussions&message.id=4...

The MineMeld TAXII DataFeed node will generate CIDRs if the miner generate IP ranges or CIDRs. Otherwise single IPs are generated.

Thanks,

luigi

[JordDurh]

Thanks!

Any pointers as to where to get details on how to change the miners from abbreviating into CIDRs? I've tried to find this but not been able to.

[lmori]

Hi @JordDurh,

the output are in charge of abbreviating ranges into CIDRs. For stdlib.taxiiDataFeed this happens automatically, for output nodes based on stdlib.feed* it happens only when you add the tr=1 parameter

[JordDurh]

Thanks!

I know you can add this to the URL when browsing, however I'm not seeing any documentation (or have not yet found) on how / if you can add this option direclty to the output config to use this as default.

Can you confirm if its possible?

[lmori]

Hi @JordDurh,

this is automatically done by default by stdlib.taxiiDataFeed nodes, but you need a parameter on the URL for the stdlib.feed* nodes. Currently there is no way to change the default of the stdlib.feed*.

Do you have a client that does not support URL with parameters ?

Thanks !

luigi 

[thanzeeer]

This is issue which I am facing. I created reference set to accept IP Address, it allways shows one IP Address. Another referrence defined for collecting URL information is pupulating and getting updated.

Regards

Thanzeer

[JordDurh]

I'm trying to use this TAXII feed within QRadar via their ThreatIntel app, this is the method detailed in your articles. I'm able to access and browse the TAXII feed, however once added no IP's get populated within reference sets (even if this is set to an alphanumeric reference set rather than an IP based one).

Am I able to modify the outputs to include the extra syntax to convert to CIDR? If so what is the contaxt to use in the output config?

[lmori]

Hi @JordDurh,

for TAXII output you don't need to add the extra syntax, the TAXII output node automatically convert IP Ranges into CIDRs.

Have you checked the stats of the output node ? How many indicators are there ?

Thanks,

luigi

[JordDurh]

There are several thousand indicators in the outputs. Both IPv4 & URL.

[lmori]

Hi @JordDurh,

could you check inside the log of the QRadar Threatintel app for errors ? 

[JordDurh]

Hi.

Im not seeing any "errors" but this is what we are seeing in the logs:

2017-04-21 10:01:10,308 [com.ibm.ThreatIntelligence] [INFO] - Polling TAXII Server [id=5, url=https://threatfeed.virtualarmour.com/taxii-discovery-service, auth_type=None, authentication=None, certfile=None, collection=VA_ThreatFeed_Taxii_IPv4, observable_type=AddressObjectType:ipv4-addr, reference_set=VA_TAXII_IPs, clear_reference_set=True, poll_interval=5, poll_initial_date=2017-04-21 09:46:53, poll_last_date=2017-04-21 10:00:12.334846, poll_status=idle, observable_count=0, observable_trend=True, total_observable_count=0, date_created=2017-04-21 09:47:28.624016, date_updated=2017-04-21 10:00:27.257629]
2017-04-21 10:01:10,340 [com.ibm.ThreatIntelligence] [INFO] - Gathering list of current collection names from https://threatfeed.virtualarmour.com/taxii-discovery-service
2017-04-21 10:01:10,340 [com.ibm.ThreatIntelligence] [INFO] - Sending Discovery request to https://threatfeed.virtualarmour.com/taxii-discovery-service
2017-04-21 10:01:10,686 [com.ibm.ThreatIntelligence] [INFO] - Found 2 TAXII servers
2017-04-21 10:01:12,149 [com.ibm.ThreatIntelligence] [INFO] - Sending Collection Information Request to https://threatfeed.virtualarmour.com/taxii-collection-management-service
2017-04-21 10:01:14,463 [com.ibm.ThreatIntelligence] [INFO] - Found 0 custom action scripts
2017-04-21 10:01:14,468 [com.ibm.ThreatIntelligence] [INFO] - Found 2 TAXII servers
2017-04-21 10:01:16,494 [com.ibm.ThreatIntelligence] [INFO] - Found collection named 'VA_ThreatFeed_Taxii_IPv4'; creating poll request for Observables since 2017-04-21 10:00:12.334846...
2017-04-21 10:01:16,495 [com.ibm.ThreatIntelligence] [INFO] - Retrieving observables from https://threatfeed.virtualarmour.com/taxii-poll-service for collection VA_ThreatFeed_Taxii_IPv4 between 2017-04-21T10:00:12Z and 2017-04-21T10:01:16Z...
2017-04-21 10:01:21,504 [com.ibm.ThreatIntelligence] [INFO] - Retrieved 0 observables from https://threatfeed.virtualarmour.com/taxii-poll-service collection VA_ThreatFeed_Taxii_IPv4 between 2017-04-21T10:00:12Z and 2017-04-21T10:01:16Z
2017-04-21 10:01:21,504 [com.ibm.ThreatIntelligence] [INFO] - No new Observable(s) were reported from https://threatfeed.virtualarmour.com/taxii-poll-service for collection VA_ThreatFeed_Taxii_IPv4
2017-04-21 10:01:21,504 [com.ibm.ThreatIntelligence] [INFO] - Poll TAXII Server ID 5 complete! Imported 0 observables into reference set VA_TAXII_IPs. 0 observables reported over the lifetime of this feed.

Its saying there are no observables in the feed, this is not correct:

ThreatFeed_Taxii_IPv4

OUTPUT

STARTED

10593

ADDED: 10593 REMOVED: 0

RX: 10593 PROCESSED: 10593 TX: 0

RX: 1659 PROCESSED: 0 TX: 0

I have also tried adding a CIDR into the IP reference set and this is not supported, only individual hosts (/32s) are supported here. 

One thing I have noticed, is that by using a custom miner which uses CSV to extract IPv4 hosts from a .txt are able to be added to a reference set via this TAXII method, however nothing coming from protoype miners are populated or seen as being "observable".

[lmori]

Hi @JordDurh,

TAXII 1.1 DataFeeds are orderd based on timestamp. When a TAXII client polls a TAXII DataFeed it typically specify a time range and all the indicators with time stamp falling in that range will be retrieved. In your case IBM QRradar is polling indicators in the range:

2017-04-21 10:01:16,495 [com.ibm.ThreatIntelligence] [INFO] - Retrieving observables from https://threatfeed.virtualarmour.com/taxii-poll-service for collection VA_ThreatFeed_Taxii_IPv4 between 2017-04-21T10:00:12Z and 2017-04-21T10:01:16Z...

The reason indicators are not retrieved is that most probably there haven't been updates in that range. There are still 10593 updates in the TAXII DataFeed but they do not fall in that time range. In QRadar you can specify an initial poll time range when you configure the feed, and after that polls will be incremental.

Yes, QRadar supports only /32 IPs in references sets.