how to write a simple miner documentation

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

how to write a simple miner documentation

L2 Linker

Hi there,

   I'm a new user, so hopefully this is a simple question.

 

I installed minemeld via source code on ubuntu 14.04 using the instructions on this page : 

https://github.com/PaloAltoNetworks/minemeld-ansible

 

 The installation went smoothly and there were no errors.

 

I then went through the exercise of writing a test miner using these instructions :  https://github.com/PaloAltoNetworks/minemeld/wiki/How-To-Write-a-Simple-Miner

 

I create the ytexample.py file in the detailed directory, replaced /opt/minemeld/local/config/committed-config.yml with the node information available in the "How-To.." webpage, and restarted the minemeld service.  From this point, I check the minemeld-engine.log file, and I see the following error:

 

minemeld-engine.log:2017-05-15T23:46:45 (14879)config._load_and_validate_config_from_file ERROR: Invalid config /opt/minemeld/local/config/committed-config.yml: Unknown node class minemeld.ft.ytexample.YTExample in testYT

 

Has anybody seen this error before?

 

Thanks...

18 REPLIES 18

L7 Applicator

Hi @vb0398,

that guide should be updated, there are 2 additional steps:

- open the file nodes.json in the main directory of the minemeld engine and add the following to the dictionary of nodes:

[...]
},
"minemeld.ft.ytexample.YTExample": {"class": "minemeld.ft.ytexample.YTExample"}
}

- run "/opt/minemeld/engine/current/bin/pip install -e /opt/minemeld/engine/core"

 

NOTE: If you are looking into creating a new Miner my suggestion is to use external extensions, they are more flexible and agile. The same code of the Miner in the guide has been packaged as an extension here:

https://github.com/PaloAltoNetworks/youtube-miner

Hi @lmori - Thanks for your reply.

 

I did those last 2 steps, and it still doesn't work.  The error this time in the minemeld-engine.log :

 

ImportError: No module named YTExample
ImportError: No module named YTExample

 

I will go ahead and try the extension route.

 

Best,

 

Hi @vb0398,

it seems that python is not able to find the YTExample module containing the Miner class.

Please could you attach the nodes.json file ? is the YTExample.py in minemeld/ft directory ?

 

Thanks,

luigi

hi @lmori,

   In your documentation, it says to name the file, 'ytexample.py' - all lowercase, and that is the name of the file in the "/opt/minemeld/engine/core/minemeld/ft" directory.  

 

Attached the nodes.json file

 

 

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!