Messed up IPv4 ranges in output

[BillC]

We are on MineMeld version 0.9.52. We had an incident with MineMeld on the 4th of March that caused a major outage for many of our users.

We have four miners feeding in IP addresses for our block list (let's call them bl-1, bl-2, bl-3 and wl-1). The first is for addresses we want to block, the next two use external threat feeds (one of them is itcertpa.IP), and the last is a white list.

All four feed into a processor (let's call it p2, with prototype stdlib.aggregatorIPv4Generic) which in turn feeds into an output node (let's call it o2). In addition, for historical reasons bl-1 feeds into p1 which feeds into o1. I don't think p1 had any other inputs (although I could be wrong).

Last Wednesday something went wrong and both output nodes contained a number of address ranges that were not in any of the feeds. These ranges appeared to represent the addresses between address entries in the feeds. For example, bl1 contained the IP addresses 200.127.121.99 and 200.194.26.234. bl-2 contained the IP address 201.14.193.151. Instead of the output of o2 containing just these three addresses, it contained:

200.127.121.99-200.127.121.99
200.127.121.100-200.194.26.233
200.194.26.234-200.194.26.234
200.194.26.235-201.14.193.150
201.14.193.151-201.14.193.151

Obviously those second and fourth lines should not have been there (there were substantially more entries like this). The very first entry in the logs for 200.194.26.235-201.14.193.150 is a TRACE / EMIT_UPDATE for source p1 and is below:

{
"comment": "user1 - 18-02-2020",
"_updated": 1583328197470,
"confidence": 100,
"_added": 1583328197470,
"share_level": "red",
"sources": [
"bl-1"
],
"first_seen": 1583328194879,
"_id": "50d9e67e-dd7c-4437-9809-5108a352c7c8",
"type": "IPv4",
"last_seen": 1583328194879
}

The comment appears to be taken from entries added on the 18th of Feb. These entries were not related to 200.194.26.234. All the incorrect entries in the log appear to contain this same comment.

I removed bl-2 and bl-3 from p-2 which seemed to also remove the wrong entries. I remove bl-1 from p-1 which basically made o-1 empty. I don't recall if there were any other feeds into p-1, but if there were I would have removed them at the same time. I later put bl-1 back into p-1 and it appeared normal.

Does anyone have any idea what might have caused this issue, or how I can find the root cause?

[lmori]

Hi @BillC,

this is weird and I guess it could have been generated by an old Miner as bl-1. Could you share more about the config? what type of Miners are you using for bl-1, bl-2 and bl-3? How do you feed bl-1?

Could you also search the logs for the string "200.194" ?

Thanks,

Luigi

[BillC]

bl-1 uses the class "minemeld.ft.local.YamlIPv4FT". It's basically where we manually add addresses to block. It has the following config:

age_out default: null
interval: 67
sudden_death: true
attributes confidence: 100
share_level: red
interval 3600

bl-2 uses the class "minemeld.ft.http.HttpFT". It has the following config (with obfuscation of the URL as we pay for it):

attributes confidence: 100
share_level: red
type: IPv4

ignore_regex ^#

source_name ET.block_ips

url https://a.domain.com/ti/v1/db-token/blackList?type=ip&listId=0&format=pan&token=12345

bl-3 uses the included itcertpa.IP prototype. Class is "minemeld.ft.http.HttpFT" and config is:

age_out default: null
interval: 600
sudden_death: true

attributes confidence: 80
direction: inbound
share_level: green
type: IPv4

ignore_regex ^#.*

indicator regex: ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$

source_name itcertpa.IP

url https://infosec.cert-pa.it/analyze/listip.txt

The logs in MimeMeld now appear to not be going back earlier than the 11th of March. However, I do have backups of the server (one week of dailies plus the first of each month). So, is there a way to extract the information from the logs using the CLI.