I have various miners. Various miners are connected to various aggregators which are inturn connected in various ways to different types of output.
Some of these miners receive RFC1918 IPv4 indicators. These are aggregated and send to outputs. I'm attempted to have one output which will contain these RFC1918 addresses while another does not.
I've created a new prototype based off of minemeld.ft.redis.RedisSet accepting only indicator types IPv4. The config includes this:
I cloned it and pointed it to clone of processor stdlib.aggregatorIPv4Outbound. The input for this processor includes my wlWhiteListIPv4 as well as my miner. This wlWhiteListIPv4 node has manual indicators which include RFC1918 CIDR blocks along with one IP address (for testing.) This, it includes 10.0.0.0/8 as well as a single host within that range. It appears to me that while the single host works in this case, the CIDR does not. With two nearly identical outputs configured where one is using the whitelist and another not, I see this single host only in the later output. However, other hosts in the 10.0.0.0/8 CIDR are in both.
How do I go about adding a miner with CIDRs as a whitelist to an output node? While in my example above I'm talking about RFC1918 addresses, there are additional cases where I want to exclude from certain output nodes the ranges collected from a miner.
I believe I see what I had wrong. Because I hwave whitelist_prefixes set to wl, the processor name itself must also begin with wl. I had only named the miner begining with wl.
So, I configured a processor with a name begining with wl and bound it to two inputs, the wl miner containing my RFC1918 addresses along with my syslog+7 days miner. This worked. Within this same config I have an identical processer with the exception of the name, it does not start with wl. The output node is identical except it points to this processor. This one has the RFC1918 addresses. The point being that with only the name of the processor changing I can see the whitelist working.
I think the following is the relevant config. I have 300+ lines in the config ATM, so I'm not posting it all.
The end result in this case is that I have PanOS syslog output feeding into MineMeld while whitelisting RFC1918 addresses. Longer term I could use external dynamic IPv4 lists, such as a list of CloudFlare addresses, as a feed into a whitelist that I aggregate against the PanOS syslog entries to generate output feeds that I can use in specific dynamic PanOS firewall rules. After enough hits from an IP, if it isn't on a known list, completely block that IP at every firewall at all locations for a period of time. This is just one example of many I'm considering.
syslogMiner7Days: inputs:  output: true prototype: minemeldlocal.syslogMiner7Days aggregatorIPv4GenericStaticWhiteList: inputs: - wlWhiteListIPv4 output: true prototype: stdlib.aggregatorIPv4Generic wlWhiteListIPv4: inputs:  output: true prototype: stdlib.listIPv4Generic localSyslog7Days: inputs: - syslogMiner7Days - wlWhiteListIPv4 output: true prototype: stdlib.localSyslog wlListIPv4Generic-testingWhiteList20171222b: inputs: - wlWhiteListIPv4 - syslogMiner7Days output: true prototype: stdlib.aggregatorIPv4Generic feedIPv4TestingWhiteList: inputs: - wlListIPv4Generic-testingWhiteList20171222b output: false prototype: minemeldlocal.feedIPv4TestingWhiteList
are you trying to create a feed out of MineMeld to be used as whitelist? or would you like to whitelist some IPs from the indicators extraced from syslog?
>> whitelist some IPs from the indicators extraced from syslog?
While not excluding them at the time of extract. Not all feeds should have these whitelist entries excluded from their final output.
Click Accept as Solution to acknowledge that the answer to your question has been provided.
The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!
These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!
The LIVEcommunity thanks you for your participation!