Mining Domains from mixed IP/Domain List

Reply
L4 Transporter

Mining Domains from mixed IP/Domain List

Hi guys,

 

I would appreciate some help withe this issue. I wanto to mine domains from a list that has IP/Domais mixed. The list has on the top a short description whose lines start with the "#" character. Thats fine, I can use the "ignore_regex ^#" to avoid it. But the list is composed by:

 

10.200.1.36 domain1.com

172.31.31.31 domain2.com

192.168.0.55 domain3.com

.

.

.

10.10.10.11 domain_n.com

 

How do I avoid the IPs and get only the domains? I belive it is not so difficult. I already got the IPs using "indicator regex: ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" .

 

Thank you in advanced.

Best regards.

L7 Applicator

Re: Mining Domains from mixed IP/Domain List

Hi @danilo.souza,

simple solution: mine the feed twice, once for the IPs and once for the domains :-)

L4 Transporter

Re: Mining Domains from mixed IP/Domain List

Hi @lmori

 

thank you for the reply. Sure, that is what I'm doing. The point is: the correct "regular expression" to get the domains (beginning include with numbers) without the space " " between the IPs and the domains itself. Until now, I was only able to get the domains including the space " ". Does the feed node know how to deal with it? The space " " at the begin is eliminated by him when producing the link to be ingested by PANOS in an EDL?

 

Thank you in advanced.

Best regards.

Highlighted
L7 Applicator

Re: Mining Domains from mixed IP/Domain List

Hi @danilo.souza,

you should be able to use the following to get the domains:

indicator:
  regex: '[\w\.]+\s(.*)'
  transform: '\1'
L3 Networker

Re: Mining Domains from mixed IP/Domain List

It's a bit more complicated, but maybe this can help as well?

 

^((?:((\d{1,3}.){3}\d{1,3}\s))(.*))$

 

I haven't tested it yet since I don't have a feed like this, but it seems to match and non-capture group on what it needs to.

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!