About KevinAS

Hey,   Yeah, you'll need to create the file /usr/share/logstash/search-minemeld-src.json with the content mentioned in my previous reply. My understanding of this was the "src_ip" would be checked against the "firstIP" and "lastIP" fields within the minemeld index. If the "src_ip" is lower than or equal to the "firstIP" AND  if the "src_ip" is greater than or equal to the "lastIP", it would take its "sources" and "confidence" values (from the event in the minemeld index) and copy these to the event that its matched.   The dissect section essentially is splitting up the indicator into two separate fields (firstIP and lastIP).   With the above method, we don't need to know the CIDR address.   Hope this helps. ... View more

Nice one! Yeah, I've been looking into this further today. I found that I don't really need the CIDR format as I can pull out the two IP addresses within the indicator and checking to see if an IP is within its range.   I have the following configured for minemeld   input { tcp { port => "1516" tags => [ "syslog" ] } } filter { if "@origin" in [message] { mutate { add_tag => "minemeld" } json { source => "message" } fingerprint { source => "@indicator" target => "[@metadata][fingerprint]" method => "MURMUR3" } dissect { mapping => { "@indicator" => "%{firstIP}-%{lastIP}" } } } } output { if "minemeld" in [tags] and [message] == "withdraw" { elasticsearch { hosts => ["localhost:9200"] index => "minemeld" action => "delete" document_id => "%{[@metadata][fingerprint]}" } } else if "minemeld" in [tags] { elasticsearch { hosts => ["localhost:9200"] index => "minemeld" document_id => "%{[@metadata][fingerprint]}" } } } This will allow the list to stay updated within Elasticsearch with any "withdraw" events being deleted.   I'm then able to query this data from other inputs such as firewalls as below   filter { elasticsearch { hosts => ["localhost:9200"] index => "minemeld" query_template => "/usr/share/logstash/search-minemeld-src.json" fields => { "sources" => "minemeld_source" "confidence" => "minemeld_confidence" } add_tag => [ "minemeld_trigger_src" ] } } I then have the following within /usr/share/logstash/search-minemeld-src.json   { "size": 1, "query": { "bool": { "filter": [ { "range": { "firstIP": { "lte": "%{[src_ip]}" }}} , { "range": { "lastIP": { "gte": "%{[src_ip]}" }}} ] } }, "_source": ["sources", "confidence"] } So for events that hit this filter (firewall traffic), the "src_ip" is checked against the minemeld index and checks if the IP address exists within each range. This then adds new fields to the firewall traffic (sources/confidence) and adds a new tag (minemeld_trigger_src). Seems to work the way I'm wanting it to, not sure if its of any use to yourself or anyone else. May give you some additional insight however.   Cheers, Kev ... View more

Previous post got removed, not sure where it went.   Managed to get this working after some additional lines in one of the files. Not the greatest fix but it does the job for now.   Had to edit the following logstash.py file (path dependent on install location) /opt/minemeld/engine/core/minemeld/ft/logstash.py Need to add an import into the file   Before: import logging import ujson import datetime import socket After: import logging import ujson import datetime import socket import netaddr Then some additional lines under the indicators section   Before: if indicator is not None: fields['@indicator'] = indicator After: if indicator is not None: fields['@indicator'] = indicator if len(indicator.split('-')) == 2: startIP = indicator.split('-')[0] endIP = indicator.split('-')[1] cidr = netaddr.iprange_to_cidrs(startIP, endIP)[0] fields['CIDR'] = str(cidr) This will produce a new field called 'CIDR'.   If anyone knows an alternative/cleaner method to achieve this, please share!   Hope this helps.   Cheers, Kev ... View more

Hey, Glad to hear you got it working! I'm working on the same problem. Ideally I'm looking to get this in a CIDR format but unsure on where to configure this whether it be on the processor stage or the logstash output. Also I'm trying to achieve this at the minemeld stage instead of using logstash filters. If anyone else can help on this part, would be helping us both out. In the meantime, I'll keep looking. Cheers, Kev ... View more