04-05-2021 01:58 AM
Apologies if this question has been asked before, I searched the board but couldn't see anything that stood out
I'm consuming the SANS/IRC list of IP addresses attributed to Internet Security Researchers, in an attempt to cut-down on false-positive threat alerts in customer networks. The Miner I wrote works fine, and pulls down about ~6700 IP addresses.
When passed into a Processor (Cloned from 'stdlib.aggregatorIPv4Generic', and of the same class 'minemeld.ft.ipop.AggregateIPv4FT') all addresses are getting sent to an output.
The problem is that because the list is gathered automatically, it has one IP per-line. Example;
102.165.30.0-102.165.30.0
102.165.30.1-102.165.30.1
102.165.30.2-102.165.30.2
102.165.30.3-102.165.30.3
...
102.165.30.255-102.165.30.255
I feel that added ~6700 entries into an EDL will be unneccessarily taxing on the firewall.
Since I'm putting in a top-level firewall rule to 'drop' packets coming from these IPs, the firewall will have to match the incoming packets IP to all ~6700 possibilities - whereas if I could consolidate the IPs (for example the above consolidates into 102.165.30.0/24) then the number of matches greatly decreases
I realise that 'aggregator' in the Processors name refers to the ability to 'aggregate' from multiple miners into one processor. But is there an ability to aggregate (/consolidate) IP addresses inside a Processor?
If this can't be done in Minemeld, then I may have to write a Python parser to pull down the list and consolidate manually - but at that point Minemeld becomes irrelevant, as if I'm hosting the output of the Python script somewhere I can just point the firewall to that instead
Thanks
