Miner to collect AWS IP

Reply
L1 Bithead

Miner to collect AWS IP

I would like to setup a new Miner to collect AWS Ips from the following http://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html, using URL https://ip-ranges.amazonaws.com/ip-ranges.json.    How would that be accomlished, I can't seem to locate a good example.


Accepted Solutions
L7 Applicator

Hi coldstone1,

there is no Miner for JSON, but it could be easily added. I have created Enachement Request #7 to track this request

View solution in original post


All Replies
L7 Applicator

Hi coldstone1,

there is no Miner for JSON, but it could be easily added. I have created Enachement Request #7 to track this request

View solution in original post

L7 Applicator
L4 Transporter

 

I can not get the AWS Ips in my output.

My miner gets the IPs, the aggregator gets the IPs, but the outout stays empty.

 

AWS-Miner:
CLASS minemeld.ft.json.SimpleJSON
PROTOTYPE aws.AMAZON
STATE STARTED
LAST RUN 2016-11-21 15:19:42 +0100 SUCCESS
# INDICATORS 457

 

AWS: aggegator:
CLASS minemeld.ft.ipop.AggregateIPv4FT
PROTOTYPE stdlib.aggregatorIPv4Inbound
STATE STARTED
# INDICATORS 457

 

AWS: Output
CLASS minemeld.ft.redis.RedisSet
PROTOTYPE stdlib.feedHCGreen
STATE STARTED
FEED BASE URL https://xxx/feeds/AWS-IPv4-output
TAGS
# INDICATORS 0

 

Am I using the wrong nodes?

L7 Applicator

Hi @mr.linus,

the problem is the output node, as AWS IPs have not share_level green (and that's an oversight) they are not accepted by feedHCGreen. Use an output node based on stdlib.feedHCWithValue please. 

L4 Transporter

Hi,

Just found it. I adjusted the miner and now I get the output.

Thanks for the tip

L0 Member

Thanks for the helpful post, is there a way for the output to be IP ranges in CIDR format, instead of it listing the first IP and the last IP within the range?

 

Thank you.

L7 Applicator

Yes, add use the following format for the URL feed:

https://<minemeld>/feeds/<aws feed>?tr=1

 

See here for additional details:

https://live.paloaltonetworks.com/t5/MineMeld-Articles/Parameters-for-the-output-feeds/ta-p/146170

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!