MineMeld age_out not withdrawing ips

Reply
PF
L1 Bithead

MineMeld age_out not withdrawing ips

I'm very new to MineMeld, and I am having issues withdrawing ip addresses from a list. 

 

The miner checks a local list, and the list has two ips in it currently. I'd like the ips to be age_out after 24 hours, even if they are still on the local list. 

 

In the logs I see TRACE / EMIT_WITHDRAW with the indicator of the ip, but then the very next log is TRACE / EMIT_UPDATE with the indicator of the ip, and the ip is never removed from the minemeld output. The miner says added 5 and removed 3, but the local list has been static. What am I missing? Thanks!

L7 Applicator

Re: MineMeld age_out not withdrawing ips

Hi @PF,

age out depends on the config and the type of output feeds. Example: standard feeds (stdlib.feed*) immediately remove expired indicators while other like taxiiDataFeed do not because their logic is different.

Could you share your config from CONFIG > EXPORT ? I can give you more details about the expected behavior.

PF
L1 Bithead

Re: MineMeld age_out not withdrawing ips

Thanks for getting back to me

 

nodes:
bunker_aggregator:
inputs:
- Bunker
output: true
prototype: stdlib.aggregatorIPv4Generic
Bunker:
inputs: []
output: true
prototype: minemeldlocal.bunker_banlist
bunker-output:
inputs:
- Bunker
output: false
prototype: stdlib.feedHCGreenWithValue

L7 Applicator

Re: MineMeld age_out not withdrawing ips

Hi @PF,

could you share more details about the minemeld.bunker_banlist prototype ? like class and full config ?

 

Thanks,

luigi

PF
L1 Bithead

Re: MineMeld age_out not withdrawing ips

--class--

minemeld.ft.http.HttpFT

--config--
age_out
default: first_seen+1d
interval: 1800
sudden_death: true
attributes
confidence: 100
direction: inbound
share_level: green
type: IPv4
ignore_regex ^#.*
interval 60
source_name bunker.banlist
url http://ip-address/banlist.txt

L7 Applicator

Re: MineMeld age_out not withdrawing ips

Hi @PF,

this is a bug, and I have already a fix for it. Would you be interested in testing the beta with the fix ?

 

luigi

PF
L1 Bithead

Re: MineMeld age_out not withdrawing ips

sure

PF
L1 Bithead

Re: MineMeld age_out not withdrawing ips

@lmori, Whats the process for testing the beta fix? I'm willing to give it a go. 

L7 Applicator

Re: MineMeld age_out not withdrawing ips

Hi @PF,

if you have installed MM from binaries (via OVA, CFN, AFM, ISO, apt repos, ...) you should subscribe your MM instance to the beta channel. Change the file /etc/minemeld-auto-updates.conf to this (basically change the value of "channel"):

{
  "minemeld-updates": {
    "baseurl": "http://minemeld-updates.panw.io/stage2",
    "channel": ["0_9", "beta0_9"]
  }
}

After that, force an update:

$ sudo -u minemeld /usr/sbin/minemeld-auto-update
PF
L1 Bithead

Re: MineMeld age_out not withdrawing ips

I changed the auto-update.conf and run the update command, but get this..

 

minemeld:/etc$ sudo -u minemeld /usr/sbin/minemeld-auto-update
Traceback (most recent call last):
File "/usr/sbin/minemeld-auto-update", line 787, in <module>
main()
File "/usr/sbin/minemeld-auto-update", line 738, in main
update_minemeld_package()
File "/usr/sbin/minemeld-auto-update", line 687, in update_minemeld_package
cache.update()
File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 418, in update
raise LockFailedException("Failed to lock %s" % lockfile)
apt.cache.LockFailedException: Failed to lock /var/lib/apt/lists/lock

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!