Miner to collect AWS IP

L1 Bithead

Miner to collect AWS IP

I would like to setup a new Miner to collect AWS Ips from the following http://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html, using URL https://ip-ranges.amazonaws.com/ip-ranges.json.    How would that be accomlished, I can't seem to locate a good example.

L7 Applicator

Re: Miner to collect AWS IP

Hi coldstone1,

there is no Miner for JSON, but it could be easily added. I have created Enachement Request #7 to track this request

L7 Applicator

Re: Miner to collect AWS IP

L4 Transporter

Re: Miner to collect AWS IP

 

I can not get the AWS Ips in my output.

My miner gets the IPs, the aggregator gets the IPs, but the outout stays empty.

 

AWS-Miner:
CLASS minemeld.ft.json.SimpleJSON
PROTOTYPE aws.AMAZON
STATE STARTED
LAST RUN 2016-11-21 15:19:42 +0100 SUCCESS
# INDICATORS 457

 

AWS: aggegator:
CLASS minemeld.ft.ipop.AggregateIPv4FT
PROTOTYPE stdlib.aggregatorIPv4Inbound
STATE STARTED
# INDICATORS 457

 

AWS: Output
CLASS minemeld.ft.redis.RedisSet
PROTOTYPE stdlib.feedHCGreen
STATE STARTED
FEED BASE URL https://xxx/feeds/AWS-IPv4-output
TAGS
# INDICATORS 0

 

Am I using the wrong nodes?

L7 Applicator

Re: Miner to collect AWS IP

Hi @mr.linus,

the problem is the output node, as AWS IPs have not share_level green (and that's an oversight) they are not accepted by feedHCGreen. Use an output node based on stdlib.feedHCWithValue please. 

L4 Transporter

Re: Miner to collect AWS IP

Hi,

Just found it. I adjusted the miner and now I get the output.

Thanks for the tip

L0 Member

Re: Miner to collect AWS IP

Thanks for the helpful post, is there a way for the output to be IP ranges in CIDR format, instead of it listing the first IP and the last IP within the range?

 

Thank you.

L7 Applicator

Re: Miner to collect AWS IP

Yes, add use the following format for the URL feed:

https://<minemeld>/feeds/<aws feed>?tr=1

 

See here for additional details:

https://live.paloaltonetworks.com/t5/MineMeld-Articles/Parameters-for-the-output-feeds/ta-p/146170

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!