Automatization of Minemeld with API

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

Automatization of Minemeld with API

L0 Member

Hi,

 

Im trying to create node using "autofocus.sampleMiner" proto using API.

 

When I try to get existing node config I get the following answer when I run /config/node/N

 

HTTP/2 200
server: nginx/1.11.3
date: Wed, 02 Jan 2019 12:19:16 GMT
content-type: application/json
content-length: 227
expires: Wed, 02 Jan 2019 12:19:15 GMT
cache-control: no-cache
strict-transport-security: max-age=15724800; includeSubDomains; preload

{
  "result": {
    "name": "af-miner",
    "properties": {
      "inputs": [],
      "output": true,
      "prototype": "autofocus.samplesMiner"
    },
    "version": "e5c8dd3c-16b5-4dbf-9798-72f8XXXXXXX"
  }
}

 

I would like to customize "autofocus query" parameter in order to create custom miners. Any option?

 

Thank you in advance,

 

Happy new year.

6 REPLIES 6

L1 Bithead

Hi,

 

I just ran this:

 

{
"name": "My af miner",
"properties": {
"prototype": "autofocus.samplesMiner",

"config": {
"attributes": {
"share_level": "red",
"confidence": "100"

}
},
"output": true,
"query": {"operator":"all","children":[{"field":"sample.malware","operator":"is","value":1}]}
},
"version": "e5c8dd3c-16b5-4dbf-9798-XXXXXXXXXXX"
}

 

And the node has been created sucessfully but now, I need to set a specific query, scope, and antifacts. Where should I put them?

 

Thank you in advance,

Hi @ITSCERT,

there is a dedicated API to add what is called "side config", i.e. parameters read by the node at runtime. The query should be added using that API.

 

Example:

 

PUT https://<minemeld>/config/data/<miner-name>_side_config

{
"query":"{\"operator\":\"all\",\"children\":[{\"field\":\"sample.malware\",\"operator\":\"is\",\"value\":1},{\"field\":\"sample.create_date\",\"operator\":\"is after\",\"value\":[\"2019-01-07\",\"2019-01-07\"]}]}",
"scope":"global",
"artifact_source":"af"
}

Hi,

 

I followed steps you detailed to me:

 

1) CreateNode running following cmds

query = '{ "name": "My_af_miner2", "properties": { "prototype": "autofocus.samplesMiner", "config": { "attributes": { "share_level": "red", "confidence": "100" } }, "output": true }, "version": "9c3d9621xxxxxxxxx."
        createNode = requests.post(URL+URI_STATUS, verify=False, auth=HTTPBasicAuth('Y, 'X'), headers={ "Content-Type": "application/json" }, data=query)
        resp = createNode.json()

 

The result is the following:

 

{'result': {'id': 87, 'version': '9c3d9621XXXXX0b-86ac-6XXXaa+0'}}

 

After create node, I verify that is correct on minemeld gui side. The only thing that needs are side_config params.

 

I run as you told me the following:

 

URI_STATUS='/config/data/My_af_miner2_side_config'
query = '{ "query":"{\"operator\":\"all\",\"children\":[{\"field\":\"sample.malware\",\"operator\":\"is\",\"value\":1},{\"field\":\"sample.create_date\",\"operator\":\"is after\",\"value\":[\"2019-01-07\",\"2019-01-07\"]}]}", "scope":"global", "artifact_source":"af" }'
modifyNode = requests.put(URL+URI_STATUS, verify=False, auth=HTTPBasicAuth('y', 'x'), headers={ "Content-Type": "application/json" }, data=query)
resp = modifyNode.json()
print(resp)

 

And the answer is:

 

{'result': 'ok'}

 

But the config of the minner has not changed.

 

I have tried also to restart engine, but i got backoff error. Any other suggestions?

 

Thank you in advance,

 

 

 

hi @lmori,

 

After PUT what you told me to test, I receive ok, but the query is not set. If I restart engine, it fails and i need to delete node and commit  to make it working.

 

 

 

Thank you in advance,

Hi @ITSCERT,

I think you have some issues with quoting in the python code. If you want to pass a string with the JSON data encoded, it should look like (note the \\):

'{"query": "{\\"operator\\":\\"all\\",\\"children\\":[{\\"field\\":\\"sample.malware\\",\\"operator\\":\\"is\\",\\"value\\":1},{\\"field\\":\\"sample.create_date\\",\\"operator\\":\\"is after\\",\\"value\\":[\\"2019-01-07\\",\\"2019-01-07\\"]}]}", "artifact_source": "af", "scope": "global"}'

 Otherwise you can also pass the dictionary to requests and requests will encode it in json for you:

query = {u'query': u'{"operator":"all","children":[{"field":"sample.malware","operator":"is","value":1},{"field":"sample.create_date","operator":"is after","value":["2019-01-07","2019-01-07"]}]}', u'artifact_source': u'af', u'scope': u'global'}

Solved!

 

Thank you @lmori

  • 5547 Views
  • 6 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!