04-05-2021 01:58 AM
Apologies if this question has been asked before, I searched the board but couldn't see anything that stood out
I'm consuming the SANS/IRC list of IP addresses attributed to Internet Security Researchers, in an attempt to cut-down on false-positive threat alerts in customer networks. The Miner I wrote works fine, and pulls down about ~6700 IP addresses.
When passed into a Processor (Cloned from 'stdlib.aggregatorIPv4Generic', and of the same class 'minemeld.ft.ipop.AggregateIPv4FT') all addresses are getting sent to an output.
The problem is that because the list is gathered automatically, it has one IP per-line. Example;
I feel that added ~6700 entries into an EDL will be unneccessarily taxing on the firewall.
Since I'm putting in a top-level firewall rule to 'drop' packets coming from these IPs, the firewall will have to match the incoming packets IP to all ~6700 possibilities - whereas if I could consolidate the IPs (for example the above consolidates into 22.214.171.124/24) then the number of matches greatly decreases
I realise that 'aggregator' in the Processors name refers to the ability to 'aggregate' from multiple miners into one processor. But is there an ability to aggregate (/consolidate) IP addresses inside a Processor?
If this can't be done in Minemeld, then I may have to write a Python parser to pull down the list and consolidate manually - but at that point Minemeld becomes irrelevant, as if I'm hosting the output of the Python script somewhere I can just point the firewall to that instead
05-31-2021 02:13 AM
Have you checked out this form Lmori, in https://live.paloaltonetworks.com/t5/minemeld-discussions/miner-to-collect-aws-ip/td-p/75925
Yes, add use the following format for the URL feed:
See here for additional details:
05-31-2021 09:22 AM
Thanks for the link, that's useful info
Unfortunately ?tr=1 won't consolidate subnets, it justr translates each line
126.96.36.199-188.8.131.52 184.108.40.206-220.127.116.11 18.104.22.168-22.214.171.124
126.96.36.199 188.8.131.52 184.108.40.206
Appreciate the reply though
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!