Syslog miner indicator

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Syslog miner indicator

L1 Bithead

Hi, i'm trying creating a indicator for SLW injections events , extract src_ip and insert into 

 

Down below the code attached to the syslogminer class stdlib.syslogMiner , unfortunaltely validate process reports errors with the only suggestions "Condition is invalid", could be either a formatting erro or a logical erro, hard to say.

 

 

conditions:
    type == 'THREAT'
    log_subtype == 'vulnerability'
    severity == 'medium'
    threat_name == 'HTTP SQL Injection Attempt'
    
indicators:
    src_ip
    
fields:

Is there any document how clearly to build indicators? I only forund some examples but not a full hsow-to

 

Regards

 

1 accepted solution

Accepted Solutions

Problem is not about YAML itself, YAML is just a way to express conditions. Conditions are boolean expressions based on simplified JMESPath expressions. You can check for a string inside a field with something like this:

 

conditions:
  - type == 'THREAT'
  - log_subtype == 'spyware'
  - contains(threat_name, 'ZeroAccess') == true
fields: null
indicators:
  - src_ip

View solution in original post

14 REPLIES 14

L7 Applicator

Hi @AlbertoZanon,

please try this:

conditions:
  - type == 'THREAT'
  - log_subtype == 'vulnerability'
  - severity == 'medium'
  - threat_name == 'HTTP SQL Injection Attempt'
fields:
indicators:
  - src_ip

We should definitely work on the documentation of syslog miner 🙂

But conditions, fields and indicators should be lists or null. In your case they were map. You can check the YAML format here: http://yaml-online-parser.appspot.com/

Hi Luigi, 

happy ne year! and thanks for the tip; so definitely was a formatting error. Good to know I have to follow full YML directives.

 

If I may just a couple of questions:

 

1- what's the way to  include substring for a specific condition? I would like build a "contains/like" string comparison (regex related) and the code down below seems not valid for minemeld while still passes the YAML online tool check. 

 

conditions:
  - threat_name =~ '[Ii]njection'

2- I also would like to track specific events by frequency and apply rule only if a specific threshold is reached. Do you suggest using aa new processor prototype or an external pythons script ?

 

Regards

 

Hi,

 

 

threat_name == 'HTTP SQL Injection Attempt'

Is no a right solution, log format contains "HTTP SQL Injection Attempt(XXX)" where XXX is the threat id, and multiple theert ids contains the same Threat name 

 

Schermata 2017-01-02 alle 16.52.57.png

 

Deinitely i need to find a regex match directive not a full equal comparison.

 

Regards

 

I am not sure if the matching rule-set supports regex, but I definitly want regex here and some more features to configure on syslog.Miner - eg. age_out: and other.

If you want to test use a ".*" at the end of the string: 

"HTTP SQL Injection Attempt.*" 

 I believe you already tried this, did you?

 

Best, Klaus

I tried to use wildcards in threat_name but did not succeed. Even if threat_name contains a string the use of ':' (colon) is not allowed. So this one would not work:

 

 

- threat_name == 'SCAN: TCP Port Scan(8001)'

We definitely need the ability to use wildcards but I fear YAML does not support it.

Hi, * and other regular expressions were already tried out with no luck.

 

I agree with you that regex and like comparison should be added and indeed xternal YAML components like YAML::Logic already got in 

Hi Luigi,

 

I have setup some miners to obtain threat intel feeds containing IOCs from our partners in addition to the minemeld base ones like Dshield, Spamhaus etc. and I have also got my PAN FW syslog being successfully processed by minemeld.  I want the IOCs from our threat feeds to be matched against my syslog for hits.  I then want to be able to investigate these hits and look at the detail of the traffic.  I tried writing an indicator rule and applying it to my panos syslog miner to match dest_ip as the indicator but must have got something wrong as the number of matches was growing too quickly.  Can you provide some doco/guidance on using indicator rules or something to help me identify IOC hits in my syslog and investigate the detail.

 

Thanks

Chad.

 

 

Problem is not about YAML itself, YAML is just a way to express conditions. Conditions are boolean expressions based on simplified JMESPath expressions. You can check for a string inside a field with something like this:

 

conditions:
  - type == 'THREAT'
  - log_subtype == 'spyware'
  - contains(threat_name, 'ZeroAccess') == true
fields: null
indicators:
  - src_ip

Hi @AlbertoZanon,

please could you post a screenshot of the syslog miner rulebase ? or you could share the file named /opt/minemeld/local/config/<miner name>_rules.yml

 

Regards,

luigi

Hi @lmori, I changed the code uing contains directive as suggested and the hit mach are starting ramping up. For sure a wiki/doc is definetely useful for this section 🙂

Schermata 2017-01-10 alle 00.40.41.png

 

 

/opt/minemeld/local/config/panos_syslog_miner_rules.ym 


- conditions: [type == 'THREAT', log_subtype == 'vulnerability', 'contains(threat_name,
''HTTP SQL Injection Attempt'') == true']
fields: null
indicators: [src_ip]
name: sql_injection
- conditions: [type == 'TRAFFIC', src_location == 'IT']
fields: null
indicators: [src_ip]
name: test_traffic
- conditions: [type == 'THREAT', src_zone == 'FIBRA', dest_port != '22']
fields: null
indicators: [src_ip, url_idx]
name: test_threat
- conditions: [type == 'THREAT', log_subtype == 'vulnerability', severity == 'informational',
'contains(threat_name, ''SSH2 Login Attempt'') == true']
fields: null
indicators: [src_ip]
name: ssh_login

 

Hi @AlbertoZanon,

rules looks fine, but if you are looking for a way to match PAN-OS syslog messages against indicators you should look into syslogMatcher node. The syslogMatcher node can also send the details of the matched PAN-OS session to logstash for archival.

syslogMiner is useful if you want to build your own EDL with indicators extracted from PAN-OS logs.

 

What scenario are you interested in ?

 

luigi

Hi,

at this stage I'm not interested in logstash + Kibana/Elasticsearch as long term repository, so syslogMiner node type seems good enough.

 

Next step is to build up a processor logic so to evaluate indicators and metrics. Any useful docs to look around?

 

My wishes are:

- extract useful indicators + fields (ie generated_time)  [done thanks to your suggestions]

- evaluate indicators against scripted conditions.

 

Some possible conditions 

1) accept indicators and create output EDL only if the same same value is seeen at least a couple of time form the same ruleid/threatid

2) accept indicatos and create output EDL only if the same value is seen from different indicators rules

 

Regards

Hi @AlbertoZanon

1) is not possible today. I would think that the best place to implement that logic is inside the syslog miner. Basically an indicator would be created only if some temporal logic is satisfied.

2) what do you mean by "different indicator rules"? different syslog miner rules or different feeds ?

 

Thanks !

luigi

L3 Networker

@lmori


Can syslog miner receive log from Palo Alto Networks NGFW platforms only?

I try send syslog CEF (correlated rule alert) from ArcSight SIEM to my hosted Minemeld. I captured traffic via tcpdump, Minemeld server get traffic from SIEM. But I not found any log on Minemeld

 

I need to send Cyber Attack alert from rule on SIEM via syslog into Minemeld.

When Minemeld receive syslog (include Source Address), It will output to stdlib.feedHCGreen.

Then PaloAlto Firewall polling via EDL. Attacker will be blocked.

 

Thank you.

  • 1 accepted solution
  • 9929 Views
  • 14 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!