how to write a simple miner documentation

[vb0398]

Hi there,

   I'm a new user, so hopefully this is a simple question.

I installed minemeld via source code on ubuntu 14.04 using the instructions on this page : 

https://github.com/PaloAltoNetworks/minemeld-ansible

 The installation went smoothly and there were no errors.

I then went through the exercise of writing a test miner using these instructions :  https://github.com/PaloAltoNetworks/minemeld/wiki/How-To-Write-a-Simple-Miner

I create the ytexample.py file in the detailed directory, replaced /opt/minemeld/local/config/committed-config.yml with the node information available in the "How-To.." webpage, and restarted the minemeld service.  From this point, I check the minemeld-engine.log file, and I see the following error:

minemeld-engine.log:2017-05-15T23:46:45 (14879)config._load_and_validate_config_from_file ERROR: Invalid config /opt/minemeld/local/config/committed-config.yml: Unknown node class minemeld.ft.ytexample.YTExample in testYT

Has anybody seen this error before?

Thanks...

[lmori]

Hi @vb0398,

that guide should be updated, there are 2 additional steps:

- open the file nodes.json in the main directory of the minemeld engine and add the following to the dictionary of nodes:

[...]
},
"minemeld.ft.ytexample.YTExample": {"class": "minemeld.ft.ytexample.YTExample"}
}

- run "/opt/minemeld/engine/current/bin/pip install -e /opt/minemeld/engine/core"

NOTE: If you are looking into creating a new Miner my suggestion is to use external extensions, they are more flexible and agile. The same code of the Miner in the guide has been packaged as an extension here:

https://github.com/PaloAltoNetworks/youtube-miner

[vb0398]

Hi @lmori - Thanks for your reply.

I did those last 2 steps, and it still doesn't work.  The error this time in the minemeld-engine.log :

ImportError: No module named YTExample
ImportError: No module named YTExample

I will go ahead and try the extension route.

Best,

[lmori]

Hi @vb0398,

it seems that python is not able to find the YTExample module containing the Miner class.

Please could you attach the nodes.json file ? is the YTExample.py in minemeld/ft directory ?

Thanks,

luigi

[vb0398]

hi @lmori,

   In your documentation, it says to name the file, 'ytexample.py' - all lowercase, and that is the name of the file in the "/opt/minemeld/engine/core/minemeld/ft" directory.  

Attached the nodes.json file