Syslog miner indicator

[AlbertoZanon]

Hi, i'm trying creating a indicator for SLW injections events , extract src_ip and insert into 

Down below the code attached to the syslogminer class stdlib.syslogMiner , unfortunaltely validate process reports errors with the only suggestions "Condition is invalid", could be either a formatting erro or a logical erro, hard to say.

conditions:
    type == 'THREAT'
    log_subtype == 'vulnerability'
    severity == 'medium'
    threat_name == 'HTTP SQL Injection Attempt'
    
indicators:
    src_ip
    
fields:

Is there any document how clearly to build indicators? I only forund some examples but not a full hsow-to

Regards

[lmori · Accepted solution]

Problem is not about YAML itself, YAML is just a way to express conditions. Conditions are boolean expressions based on simplified JMESPath expressions. You can check for a string inside a field with something like this:

conditions:
  - type == 'THREAT'
  - log_subtype == 'spyware'
  - contains(threat_name, 'ZeroAccess') == true
fields: null
indicators:
  - src_ip

[lmori]

Hi @AlbertoZanon,

please try this:

conditions:
  - type == 'THREAT'
  - log_subtype == 'vulnerability'
  - severity == 'medium'
  - threat_name == 'HTTP SQL Injection Attempt'
fields:
indicators:
  - src_ip

We should definitely work on the documentation of syslog miner :-)

But conditions, fields and indicators should be lists or null. In your case they were map. You can check the YAML format here: http://yaml-online-parser.appspot.com/

[AlbertoZanon]

Hi Luigi, 

happy ne year! and thanks for the tip; so definitely was a formatting error. Good to know I have to follow full YML directives.

If I may just a couple of questions:

1- what's the way to  include substring for a specific condition? I would like build a "contains/like" string comparison (regex related) and the code down below seems not valid for minemeld while still passes the YAML online tool check. 

conditions:
  - threat_name =~ '[Ii]njection'

2- I also would like to track specific events by frequency and apply rule only if a specific threshold is reached. Do you suggest using aa new processor prototype or an external pythons script ?

Regards

[AlbertoZanon]

Hi,

threat_name == 'HTTP SQL Injection Attempt'

Is no a right solution, log format contains "HTTP SQL Injection Attempt(XXX)" where XXX is the threat id, and multiple theert ids contains the same Threat name 

Deinitely i need to find a regex match directive not a full equal comparison.

Regards

[KlausGroeger]

I am not sure if the matching rule-set supports regex, but I definitly want regex here and some more features to configure on syslog.Miner - eg. age_out: and other.

If you want to test use a ".*" at the end of the string: 

"HTTP SQL Injection Attempt.*" 

 I believe you already tried this, did you?

Best, Klaus

[KlausGroeger]

I tried to use wildcards in threat_name but did not succeed. Even if threat_name contains a string the use of ':' (colon) is not allowed. So this one would not work:

- threat_name == 'SCAN: TCP Port Scan(8001)'

We definitely need the ability to use wildcards but I fear YAML does not support it.

[AlbertoZanon]

Hi, * and other regular expressions were already tried out with no luck.

I agree with you that regex and like comparison should be added and indeed xternal YAML components like YAML::Logic already got in 

[chadwalker]

Hi Luigi,

I have setup some miners to obtain threat intel feeds containing IOCs from our partners in addition to the minemeld base ones like Dshield, Spamhaus etc. and I have also got my PAN FW syslog being successfully processed by minemeld.  I want the IOCs from our threat feeds to be matched against my syslog for hits.  I then want to be able to investigate these hits and look at the detail of the traffic.  I tried writing an indicator rule and applying it to my panos syslog miner to match dest_ip as the indicator but must have got something wrong as the number of matches was growing too quickly.  Can you provide some doco/guidance on using indicator rules or something to help me identify IOC hits in my syslog and investigate the detail.

Thanks

Chad.

[lmori · Accepted solution]

Problem is not about YAML itself, YAML is just a way to express conditions. Conditions are boolean expressions based on simplified JMESPath expressions. You can check for a string inside a field with something like this:

conditions:
  - type == 'THREAT'
  - log_subtype == 'spyware'
  - contains(threat_name, 'ZeroAccess') == true
fields: null
indicators:
  - src_ip

[lmori]

Hi @AlbertoZanon,

please could you post a screenshot of the syslog miner rulebase ? or you could share the file named /opt/minemeld/local/config/<miner name>_rules.yml

Regards,

luigi