IP addresses disappearing from miner

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

IP addresses disappearing from miner

L2 Linker

Hello,

 

I've created a miner to add IP address based on stdlib.listIPv4Generic and class minemeld.ft.local.YamlIPv4FT. Default configuration (just cloned).

 

This week, some IP addresses just disappeared from this miner.

We added an IP address to the miner last 21st, on the 22nd of this month the IP address disappeared from the miner. Older indicators were not removed from the miner.

When I checked the logs, I found a EMIT_WITHDRAW event:

{

    "comment": " Win32.Conficker.Cp2p",

    "_age_out": 1511357349093,

    "confidence": 100,

    "_withdrawn": 1511357349181,

    "direction": "inbound",

    "share_level": "red",

    "_last_run": 1511278036128,

    "sources": [

        "feed_IPv4"

    ],

    "first_seen": 1511277006844,

    "type": "IPv4",

    "last_seen": 1511277006844

}

 

Here is the EMIT_UPDATE event for this indicator:

{

    "comment": "Win32.Conficker.Cp2p",

    "_age_out": 4294967295000,

    "confidence": 100,

    "direction": "inbound",

    "share_level": "red",

    "_last_run": 1511277006844,

    "sources": [

        "feed_IPv4"

    ],

    "first_seen": 1511277006844,

    "type": "IPv4",

    "last_seen": 1511277006844

}

 

 

 

 

 

Here is another  IP address that had the same issue:

{

    "comment": "North Korean Trojan: Volgmer",

    "_age_out": 1511181387647,

    "confidence": 100,

    "_withdrawn": 1511181387873,

    "share_level": "red",

    "_last_run": 1511027230747,

    "sources": [

        "feed_IPv4"

    ],

    "first_seen": 1510845540625,

    "type": "IPv4",

    "last_seen": 1510845540625

}

 

Apparently, 200 IPv4 were removed from this miner and I can’t understand why. As I mentioned, older indicators were not removed from the miner.

 

Can you please help?

I am attaching here all events involved in one indicator that went away from the miner.

1 accepted solution

Accepted Solutions

Hi @alyssonalmeida,

yes, it's a limitation/bug for the local Miner - something we are phasing out. There is already a new version of the local Miner called "stdlib.localDB" Miner where this limitation has been removed, with additional features (TTL per indicator) and improved API and scalability. My suggestion is to switch from the old local Miner to the new one.

 

luigi

View solution in original post

7 REPLIES 7

L2 Linker

I understand now what happened. It is a bug in MineMeld (in my opinion)

 

Let’s say you have two users connected to MineMeld.

 

The first user opens one miner to add an indicator to it. At the same time, a second user opens the same miner. So, they are listing the same indicators.

Now, let’s say that the first user adds an indicator to the miner, but the second user doest refresh the page and adds an indicator just after the first user.

 

It happens that MineMeld will withdraw the first indicator added by the first user because the browser of the last user (second user) didn’t list that first indicator.

I did the test here and I could reproduce this issue.

 

 

So, I guess someone in my company left MineMeld opened and didn’t refresh the browser before adding indicators to MineMeld.  We lost 200 indicators because of that.

 

 

 

 Can someone from MineMeld team confirm that this is a bug?

Hi @alyssonalmeida,

yes, it's a limitation/bug for the local Miner - something we are phasing out. There is already a new version of the local Miner called "stdlib.localDB" Miner where this limitation has been removed, with additional features (TTL per indicator) and improved API and scalability. My suggestion is to switch from the old local Miner to the new one.

 

luigi

Thank you, @lmori.

 

Is there any easy way to export the indicators from the old miner to the new one?

Trying to save time here 🙂

 

Is it possible to import a CSV into the miner?

 

Yes, that's exactly the best way.

With the stdlib.localDB miner what is the default age out setting? I do not see it in the config. Is it possible to adjust the default age out for indicators that do not come in with a ttl?

  • 1 accepted solution
  • 7916 Views
  • 7 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!