topic Re: Same Version Differnt Results in General Topics

Same Version Differnt Results

0isac0 — Thu, 31 Aug 2017 14:34:37 GMT

I have two instances of MineMeld, both built off the instructions at https://live.paloaltonetworks.com/t5/MineMeld-Articles/Manually-install-MineMeld-on-Ubuntu-Server-14-04/ta-p/98454

My dev instance built in June and reporting version 0.9.40 appears to be functioning correctly

My prod instance built in Aug and reporting version 0.9.40 appears to be simply echoing the raw data as the output.

This does not occur on all datafeeds. In particular it does occur on a cloned protoype for IPv4, stdlib.aggregatorIPv4Generic.

The source miner nodes all report the same number of IOCs. Once the data hits the processor node the dev instance reports 139 IOCs while the prod reports 1236 IOCs.

I have attempted to verify permissions, compare files etc. The last modification was copying the same custom prototypes file minemeldlocal.yml to both instances.

What am I missing?

Re: Same Version Differnt Results

xhoms — Thu, 31 Aug 2017 15:24:58 GMT

Thinks I'd check:

Grab configuration files from both instances (Config -> Export) and double check they're exactly the same.
Deep dive into the filter configuration section of all processors and output nodes. Different share_levels and confidence matching conditions could explain it.
Verify the share_level attribute in the miners is the expected one.

Re: Same Version Differnt Results

0isac0 — Thu, 31 Aug 2017 16:01:56 GMT

xhoms - Yep, had done all that which is what was really confusing me.

Update: I stopped services, deleted all exisiting data files in /opt/minemeld/local/data, restarted services.
Now the two instances are reporting the same numbers across the board.

Am I possibly correct in assuming there was some difference in learned confidence values as the dev instance had been runnning for an additional 60ish days, or some other built in logic?

Re: Same Version Differnt Results

xhoms — Thu, 31 Aug 2017 16:10:16 GMT

Depending on the age_out policy of a miner you can expect the instance running for more time to keep indicators that have never been seen by the newer instance. But if the number of indicators at the miner level between instances is the same then I would assume they all have the same confidence values.

A common mistake is to "clone" a miner prototype from the library that has "share_level = red" and attaching it to a processor with a filter that only accepts indicators in the "share_level = green"