MineMeld age_out not withdrawing ips

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.

MineMeld age_out not withdrawing ips

L1 Bithead

I'm very new to MineMeld, and I am having issues withdrawing ip addresses from a list. 

 

The miner checks a local list, and the list has two ips in it currently. I'd like the ips to be age_out after 24 hours, even if they are still on the local list. 

 

In the logs I see TRACE / EMIT_WITHDRAW with the indicator of the ip, but then the very next log is TRACE / EMIT_UPDATE with the indicator of the ip, and the ip is never removed from the minemeld output. The miner says added 5 and removed 3, but the local list has been static. What am I missing? Thanks!

12 REPLIES 12

L7 Applicator

Hi @PF,

age out depends on the config and the type of output feeds. Example: standard feeds (stdlib.feed*) immediately remove expired indicators while other like taxiiDataFeed do not because their logic is different.

Could you share your config from CONFIG > EXPORT ? I can give you more details about the expected behavior.

Thanks for getting back to me

 

nodes:
bunker_aggregator:
inputs:
- Bunker
output: true
prototype: stdlib.aggregatorIPv4Generic
Bunker:
inputs: []
output: true
prototype: minemeldlocal.bunker_banlist
bunker-output:
inputs:
- Bunker
output: false
prototype: stdlib.feedHCGreenWithValue

L7 Applicator

Hi @PF,

could you share more details about the minemeld.bunker_banlist prototype ? like class and full config ?

 

Thanks,

luigi

--class--

minemeld.ft.http.HttpFT

--config--
age_out
default: first_seen+1d
interval: 1800
sudden_death: true
attributes
confidence: 100
direction: inbound
share_level: green
type: IPv4
ignore_regex ^#.*
interval 60
source_name bunker.banlist
url http://ip-address/banlist.txt

L7 Applicator

Hi @PF,

this is a bug, and I have already a fix for it. Would you be interested in testing the beta with the fix ?

 

luigi

sure

@lmori, Whats the process for testing the beta fix? I'm willing to give it a go. 

L7 Applicator

Hi @PF,

if you have installed MM from binaries (via OVA, CFN, AFM, ISO, apt repos, ...) you should subscribe your MM instance to the beta channel. Change the file /etc/minemeld-auto-updates.conf to this (basically change the value of "channel"):

{
  "minemeld-updates": {
    "baseurl": "http://minemeld-updates.panw.io/stage2",
    "channel": ["0_9", "beta0_9"]
  }
}

After that, force an update:

$ sudo -u minemeld /usr/sbin/minemeld-auto-update

I changed the auto-update.conf and run the update command, but get this..

 

minemeld:/etc$ sudo -u minemeld /usr/sbin/minemeld-auto-update
Traceback (most recent call last):
File "/usr/sbin/minemeld-auto-update", line 787, in <module>
main()
File "/usr/sbin/minemeld-auto-update", line 738, in main
update_minemeld_package()
File "/usr/sbin/minemeld-auto-update", line 687, in update_minemeld_package
cache.update()
File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 418, in update
raise LockFailedException("Failed to lock %s" % lockfile)
apt.cache.LockFailedException: Failed to lock /var/lib/apt/lists/lock

L7 Applicator

Hi @PF,

most probably you have a process working on the apt database. See here: https://askubuntu.com/questions/335794/could-not-get-lock-var-lib-apt-lists-lock

 

Thanks,

luigi 

L2 Linker

Hi,

Can i drop IOCs from taxii feed after 90 days?

My source(soltra) is not providing that option i am hoping minemeld can do this?

Hi @akapucu,

 

yes. It can. You just have to adjust the aging policy of your miner. Details at https://live.paloaltonetworks.com/t5/MineMeld-Articles/Configuring-nodes/ta-p/77185.

 

The "TaxiiClient" class extends the "BasePollerFT" which means it inherits all its capabilities including the indicators aging out engine.

  • 6791 Views
  • 12 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!