topic Re: Minemeld Regex in General Topics

Minemeld Regex

bokeke — Wed, 05 Jul 2017 18:16:56 GMT

I want to only use the url portion of this feed ignoring the protocol portion http://

https://ransomwaretracker.abuse.ch/downloads/LY_DS_URLBL.txt

My regex is below:

regex: ^(http:\/\/)(.*)
transform: \2

This works fine outside Minemeld as python regex. However, Minemeld uses the full match which includes the protocol portion not just group 2 of the match in my aggregated feed.

Re: Minemeld Regex

lmori — Fri, 07 Jul 2017 09:10:50 GMT

Hi @bokeke,

you should do something like this (tested):

age_out:
    default: null
    sudden_death: true
attributes:
    confidence: 100
    share_level: green
    type: URL
ignore_regex: ^#
indicator:
    regex: ^(http[s]*:\/\/)(.*)
    transform: \2
interval: 300
source_name: ransomwaretracker.LY_DS_URLBL
url: https://ransomwaretracker.abuse.ch/downloads/LY_DS_URLBL.txt

Please note that if you are intersted in removing protocols from the output feed, you can also use the "v=panosurl" URL parameter to do that.

Re: Minemeld Regex

bokeke — Fri, 07 Jul 2017 16:44:56 GMT

Thanks imori Your regex under indicator works.

Re: Minemeld Regex

Carlos_Gomes — Thu, 11 Jul 2019 19:15:45 GMT

@lmori Has anyone got this working for a taxii client feed instead of a csv file feed?