MineMeld Discussions

Reply
L0 Member
Posts: 3
Registered: ‎03-03-2016

Automatization of Minemeld with API

Hi,

 

Im trying to create node using "autofocus.sampleMiner" proto using API.

 

When I try to get existing node config I get the following answer when I run /config/node/N

 

HTTP/2 200
server: nginx/1.11.3
date: Wed, 02 Jan 2019 12:19:16 GMT
content-type: application/json
content-length: 227
expires: Wed, 02 Jan 2019 12:19:15 GMT
cache-control: no-cache
strict-transport-security: max-age=15724800; includeSubDomains; preload

{
  "result": {
    "name": "af-miner",
    "properties": {
      "inputs": [],
      "output": true,
      "prototype": "autofocus.samplesMiner"
    },
    "version": "e5c8dd3c-16b5-4dbf-9798-72f8XXXXXXX"
  }
}

 

I would like to customize "autofocus query" parameter in order to create custom miners. Any option?

 

Thank you in advance,

 

Happy new year.

L1 Bithead
Posts: 5
Registered: ‎10-16-2018

Re: Automatization of Minemeld with API

Hi,

 

I just ran this:

 

{
"name": "My af miner",
"properties": {
"prototype": "autofocus.samplesMiner",

"config": {
"attributes": {
"share_level": "red",
"confidence": "100"

}
},
"output": true,
"query": {"operator":"all","children":[{"field":"sample.malware","operator":"is","value":1}]}
},
"version": "e5c8dd3c-16b5-4dbf-9798-XXXXXXXXXXX"
}

 

And the node has been created sucessfully but now, I need to set a specific query, scope, and antifacts. Where should I put them?

 

Thank you in advance,

L7 Applicator
Posts: 1,036
Registered: ‎03-03-2011

Re: Automatization of Minemeld with API

[ Edited ]

Hi @ITSCERT,

there is a dedicated API to add what is called "side config", i.e. parameters read by the node at runtime. The query should be added using that API.

 

Example:

 

PUT https://<minemeld>/config/data/<miner-name>_side_config

{
"query":"{\"operator\":\"all\",\"children\":[{\"field\":\"sample.malware\",\"operator\":\"is\",\"value\":1},{\"field\":\"sample.create_date\",\"operator\":\"is after\",\"value\":[\"2019-01-07\",\"2019-01-07\"]}]}",
"scope":"global",
"artifact_source":"af"
}
L1 Bithead
Posts: 5
Registered: ‎10-16-2018

Re: Automatization of Minemeld with API

[ Edited ]

Hi,

 

I followed steps you detailed to me:

 

1) CreateNode running following cmds

query = '{ "name": "My_af_miner2", "properties": { "prototype": "autofocus.samplesMiner", "config": { "attributes": { "share_level": "red", "confidence": "100" } }, "output": true }, "version": "9c3d9621xxxxxxxxx."
        createNode = requests.post(URL+URI_STATUS, verify=False, auth=HTTPBasicAuth('Y, 'X'), headers={ "Content-Type": "application/json" }, data=query)
        resp = createNode.json()

 

The result is the following:

 

{'result': {'id': 87, 'version': '9c3d9621XXXXX0b-86ac-6XXXaa+0'}}

 

After create node, I verify that is correct on minemeld gui side. The only thing that needs are side_config params.

 

I run as you told me the following:

 

URI_STATUS='/config/data/My_af_miner2_side_config'
query = '{ "query":"{\"operator\":\"all\",\"children\":[{\"field\":\"sample.malware\",\"operator\":\"is\",\"value\":1},{\"field\":\"sample.create_date\",\"operator\":\"is after\",\"value\":[\"2019-01-07\",\"2019-01-07\"]}]}", "scope":"global", "artifact_source":"af" }'
modifyNode = requests.put(URL+URI_STATUS, verify=False, auth=HTTPBasicAuth('y', 'x'), headers={ "Content-Type": "application/json" }, data=query)
resp = modifyNode.json()
print(resp)

 

And the answer is:

 

{'result': 'ok'}

 

But the config of the minner has not changed.

 

I have tried also to restart engine, but i got backoff error. Any other suggestions?

 

Thank you in advance,

 

 

 

L1 Bithead
Posts: 5
Registered: ‎10-16-2018

Re: Automatization of Minemeld with API

hi @lmori,

 

After PUT what you told me to test, I receive ok, but the query is not set. If I restart engine, it fails and i need to delete node and commit  to make it working.

 

 

 

Thank you in advance,

Highlighted
L7 Applicator
Posts: 1,036
Registered: ‎03-03-2011

Re: Automatization of Minemeld with API

Hi @ITSCERT,

I think you have some issues with quoting in the python code. If you want to pass a string with the JSON data encoded, it should look like (note the \\):

'{"query": "{\\"operator\\":\\"all\\",\\"children\\":[{\\"field\\":\\"sample.malware\\",\\"operator\\":\\"is\\",\\"value\\":1},{\\"field\\":\\"sample.create_date\\",\\"operator\\":\\"is after\\",\\"value\\":[\\"2019-01-07\\",\\"2019-01-07\\"]}]}", "artifact_source": "af", "scope": "global"}'

 Otherwise you can also pass the dictionary to requests and requests will encode it in json for you:

query = {u'query': u'{"operator":"all","children":[{"field":"sample.malware","operator":"is","value":1},{"field":"sample.create_date","operator":"is after","value":["2019-01-07","2019-01-07"]}]}', u'artifact_source': u'af', u'scope': u'global'}
L1 Bithead
Posts: 5
Registered: ‎10-16-2018

Re: Automatization of Minemeld with API

Solved!

 

Thank you @lmori

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!