MineMeld Articles

Parameters for the output feeds

by lmori on ‎03-06-2017 10:04 AM - edited on ‎08-25-2017 04:50 PM by xhoms (13,029 Views)

Each output node based on class minemeld.ft.redis.RedisSet has associated a feed accessible via the MineMeld API. The URL of the feed is shown in the node view.

feed-sshot.png

 

Additional parameters

You can use additional parameters on the feed URL to change the output format or the entry returned from the feed. You can combine multiple parameters in the same URL.

Parameter Description Example
(none) default format, the list of indicators is retrieved

https://minemeld/feeds/feed1

 

Result

1.10.16.0-1.10.31.255
1.116.0.0-1.119.4.98
1.119.12.116-1.119.255.255
1.119.4.100-1.119.12.114
1.32.128.0-1.32.191.255
101.192.0.0-101.195.255.255
101.202.0.0-101.202.255.255
101.203.128.0-101.203.159.255
101.248.0.0-101.249.235.117
101.249.235.119-101.249.255.255
101.252.0.0-101.253.255.255
103.16.76.0-103.16.76.255
103.2.44.0-103.2.47.255
[...]
s=<N> s=<N> retrieves entries starting from entry number N.

https://minemeld/feeds/feed1?s=3

 

Result

1.119.4.100-1.119.12.114
1.32.128.0-1.32.191.255
101.192.0.0-101.195.255.255
101.202.0.0-101.202.255.255
101.203.128.0-101.203.159.255
101.248.0.0-101.249.235.117
101.249.235.119-101.249.255.255
101.252.0.0-101.253.255.255
103.16.76.0-103.16.76.255
103.2.44.0-103.2.47.255
[...]
n=<M> n=<M> retrieves M entries from the feed. Can be combined with parameter s to select a subsect of the feed.

https://minemeld/feeds/feed1?s=3&n=2

 

Result

1.119.4.100-1.119.12.114
1.32.128.0-1.32.191.255
tr=1 translate IP ranges into CIDRs. This can be used also with v=json and v=csv.

https://minemeld/feeds/feed1?tr=1

 

Result

1.10.16.0/20
1.116.0.0/15
1.118.0.0/16
1.119.0.0/22
1.119.4.0/26
1.119.4.64/27
1.119.4.96/31
[...]
v=json

returns the indicator list in JSON format.

 

Note that the value of the indicator is returned only if the value flag is set in the prototype.

https://minemeld/feeds/feed1?v=json

 

Result

[
{"indicator":"1.10.16.0-1.10.31.255","value":{[...]
v=json-seq

returns the indicator list in JSON-SEQ format.

 

Note that the value of the indicator is returned only if the value flag is set in the prototype.

https://minemeld/feeds/feed1?v=json-seq

v=panosurl if the feed contains URL indicators, they are returned in a format compatible with PAN-OS URL EDLs https://minemeld/feeds/feed1?v=panosurl
v=mwg returns the indicator list in a McAfee Web Gateway compatible format as described in https://community.mcafee.com/docs/DOC-5208

https://minemeld/feeds/feed1?v=mwg

 

Result

type=string
"iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com" "WanaCrypt0r_Miner"
"www.iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com" "WanaCrypt0r_Miner"
"iuyuip.com" "WanaCrypt0r_Miner"
"oaagmx.com" "WanaCrypt0r_Miner"

In the case the indicator feed is composed by IP addresses then you can modify the output type with the t=ip additional attribute

 

Example:

https://minemeld/feeds/feed1?v=mwg&t=ip

 

Result

type=ip
"82.195.75.101" "WanaCrypt0r_Miner"
"1.211.23.1" "WanaCrypt0r_Miner"
"1.211.23.152" "WanaCrypt0r_Miner"
"1.211.23.2" "WanaCrypt0r_Miner"
"101.159.183.1" "WanaCrypt0r_Miner"
"101.52.197.161" "WanaCrypt0r_Miner"
"102.224.162.252" "WanaCrypt0r_Miner"
"11.175.27.1" "WanaCrypt0r_Miner"
v=bluecoat returns the indicator list in a BlueCoat Local List format as described in this Technical Brief document

Optional attributes:

  • cd=<category_name> (Category Default): Default Category where the indicators will be placed to
  • ca=<attribute_name> (Category Attribute): The indicator might have an additional attribute with a list of strings describing the categories it should be listed on.

Example:

https://minemeld/feeds/feed1?v=bluecoat&cd=FROMAUTOFOCUS&ca=bc_category

 

Result

define category MM_MALWARE
phishyou.foobar.com
end
define category FROMAUTOFOCUS
iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com
www.iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com
iuyuip.com
oaagmx.com
end
v=csv

returns the indicator list in CSV format.

 

The list of the attributes is specified by using the parameter f one or more times. The default name of the column is the name of the attribute, to specify a column name add |column_name in the f parameter value.

 

The h parameter can be used to control the generation of the CSV header. When unset (h=0) the header is not generated. Default: set.

 

Encoding is utf-8. By default no UTF-8 BOM is generated. If ubom=1 is added to the parameter list, a UTF-8 BOM is generated for compatibility.

 

https://minemeld/feeds/feed1?v=csv&f=confidence&f=sources|feeds&f=indicator|clientip&f=dshield_email

 

confidence,feeds,clientip,dshield_email
100,dshield.block,104.193.252.0/24,abuse@king-servers.com
...

 

Comments
by paul_w
‎07-12-2017 05:49 AM - edited ‎07-12-2017 06:02 AM

Hello -

 

I have created an EDL in PANOS 8.0.0 using a feed from Minemeld 0.9.40, when I commit I receive the following message:

 

EDL(vsys1/Skype-IPv4 ip) Downloaded file is not a text file.

 

Does anyone know how to correct the error ?

 

Thanks

by lmori
on ‎07-12-2017 06:03 AM

Hi @paul_w,

could you open discussion under MineMeld discussions about this issue ? 99% probability this is a connectivity issue or certificate issue, I know the PAN-OS error message is misleading.

by MohammedS
on ‎02-12-2018 11:03 PM

When I am trying to download feeds using Curl script and below API URL, only IP address information is getting, not confidence value and sources detail.

 

https://minemeld/feeds/feed1?tr=1&v=csv&f=indicator|clientip&f=confidence&f=sources|feeds

 

Does anyone know how to fix the issue?

by xhoms
on ‎02-15-2018 10:53 PM

@MohammedS,

 

you must be working on a output node whose prototype do not enable the storage of "values" (metadata of the indicator).

 

If you're using nodes from the standard library then chose the ones with the "WithValue" suffix in the name.2018-02-16_07-49-16.png

 

If you're creating your own prototypes then make sure you enable the "store_value" configuration attribute.

 

2018-02-16_07-49-53.png

 

 

 

 

 

 

Ask Questions Get Answers Join the Live Community
Labels
Contributors