Parameters for the output feeds

Printer Friendly Page

Each output node based on class minemeld.ft.redis.RedisSet has associated a feed accessible via the MineMeld API. The URL of the feed is shown in the node view.

feed-sshot.png

 

Additional parameters

You can use additional parameters on the feed URL to change the output format or the entry returned from the feed. You can combine multiple parameters in the same URL.

Parameter Description Example
(none) default format, the list of indicators is retrieved

https://minemeld/feeds/feed1

 

Result

1.10.16.0-1.10.31.255
1.116.0.0-1.119.4.98
1.119.12.116-1.119.255.255
1.119.4.100-1.119.12.114
1.32.128.0-1.32.191.255
101.192.0.0-101.195.255.255
101.202.0.0-101.202.255.255
101.203.128.0-101.203.159.255
101.248.0.0-101.249.235.117
101.249.235.119-101.249.255.255
101.252.0.0-101.253.255.255
103.16.76.0-103.16.76.255
103.2.44.0-103.2.47.255
[...]
s=<N> s=<N> retrieves entries starting from entry number N.

https://minemeld/feeds/feed1?s=3

 

Result

1.119.4.100-1.119.12.114
1.32.128.0-1.32.191.255
101.192.0.0-101.195.255.255
101.202.0.0-101.202.255.255
101.203.128.0-101.203.159.255
101.248.0.0-101.249.235.117
101.249.235.119-101.249.255.255
101.252.0.0-101.253.255.255
103.16.76.0-103.16.76.255
103.2.44.0-103.2.47.255
[...]
n=<M> n=<M> retrieves M entries from the feed. Can be combined with parameter s to select a subsect of the feed.

https://minemeld/feeds/feed1?s=3&n=2

 

Result

1.119.4.100-1.119.12.114
1.32.128.0-1.32.191.255
tr=1 translate IP ranges into CIDRs. This can be used also with v=json and v=csv.

https://minemeld/feeds/feed1?tr=1

 

Result

1.10.16.0/20
1.116.0.0/15
1.118.0.0/16
1.119.0.0/22
1.119.4.0/26
1.119.4.64/27
1.119.4.96/31
[...]
v=json

returns the indicator list in JSON format.

 

Note that the value of the indicator is returned only if the value flag is set in the prototype.

https://minemeld/feeds/feed1?v=json

 

Result

[
{"indicator":"1.10.16.0-1.10.31.255","value":{[...]
v=json-seq

returns the indicator list in JSON-SEQ format.

 

Note that the value of the indicator is returned only if the value flag is set in the prototype.

https://minemeld/feeds/feed1?v=json-seq

v=panosurl

if the feed contains URL indicators, they are returned in a format compatible with PAN-OS URL EDLs.

Optional attributes:

  • di=<anything> Drop Invalid entries. If an URL entry is not compliant with PAN-OS EDL URL format the entry is dropped instead of being rewritten
  • sp=<anythin> Strip Port. Ignores URL entries with ports instead of rewriting them

https://minemeld/feeds/feed1?v=panosurl

v=mwg returns the indicator list in a McAfee Web Gateway compatible format as described in https://community.mcafee.com/docs/DOC-5208

https://minemeld/feeds/feed1?v=mwg

 

Result

type=string
"iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com" "WanaCrypt0r_Miner"
"www.iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com" "WanaCrypt0r_Miner"
"iuyuip.com" "WanaCrypt0r_Miner"
"oaagmx.com" "WanaCrypt0r_Miner"

In the case the indicator feed is composed by IP addresses then you can modify the output type with the t=ip additional attribute

 

Example:

https://minemeld/feeds/feed1?v=mwg&t=ip

 

Result

type=ip
"82.195.75.101" "WanaCrypt0r_Miner"
"1.211.23.1" "WanaCrypt0r_Miner"
"1.211.23.152" "WanaCrypt0r_Miner"
"1.211.23.2" "WanaCrypt0r_Miner"
"101.159.183.1" "WanaCrypt0r_Miner"
"101.52.197.161" "WanaCrypt0r_Miner"
"102.224.162.252" "WanaCrypt0r_Miner"
"11.175.27.1" "WanaCrypt0r_Miner"
v=bluecoat returns the indicator list in a BlueCoat Local List format as described in this Technical Brief document

Optional attributes:

  • cd=<category_name> (Category Default): Default Category where the indicators will be placed to
  • ca=<attribute_name> (Category Attribute): The indicator might have an additional attribute with a list of strings describing the categories it should be listed on.

Example:

https://minemeld/feeds/feed1?v=bluecoat&cd=FROMAUTOFOCUS&ca=bc_category

 

Result

define category MM_MALWARE
phishyou.foobar.com
end
define category FROMAUTOFOCUS
iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com
www.iuqerfsodp9ifjaposdfjhgosurijfaewrwergwea.com
iuyuip.com
oaagmx.com
end
v=csv

returns the indicator list in CSV format.

 

The list of the attributes is specified by using the parameter f one or more times. The default name of the column is the name of the attribute, to specify a column name add |column_name in the f parameter value.

 

The h parameter can be used to control the generation of the CSV header. When unset (h=0) the header is not generated. Default: set.

 

Encoding is utf-8. By default no UTF-8 BOM is generated. If ubom=1 is added to the parameter list, a UTF-8 BOM is generated for compatibility.

 

https://minemeld/feeds/feed1?v=csv&f=confidence&f=sources|feeds&f=indicator|clientip&f=dshield_email

 

confidence,feeds,clientip,dshield_email
100,dshield.block,104.193.252.0/24,abuse@king-servers.com
...

 

Tags (3)
Comments

Hello -

 

I have created an EDL in PANOS 8.0.0 using a feed from Minemeld 0.9.40, when I commit I receive the following message:

 

EDL(vsys1/Skype-IPv4 ip) Downloaded file is not a text file.

 

Does anyone know how to correct the error ?

 

Thanks

Hi @paul_w,

could you open discussion under MineMeld discussions about this issue ? 99% probability this is a connectivity issue or certificate issue, I know the PAN-OS error message is misleading.

When I am trying to download feeds using Curl script and below API URL, only IP address information is getting, not confidence value and sources detail.

 

https://minemeld/feeds/feed1?tr=1&v=csv&f=indicator|clientip&f=confidence&f=sources|feeds

 

Does anyone know how to fix the issue?

@MohammedS,

 

you must be working on a output node whose prototype do not enable the storage of "values" (metadata of the indicator).

 

If you're using nodes from the standard library then chose the ones with the "WithValue" suffix in the name.2018-02-16_07-49-16.png

 

If you're creating your own prototypes then make sure you enable the "store_value" configuration attribute.

 

2018-02-16_07-49-53.png

 

 

 

 

 

 

@lmori I see that a couple of additional output formats have been added. Is it possible to create an output format for Bro/Zeek Intel Framework? The CIDRs output format gets close but Bro doesn't seem to be able to accept anything except individual IP addresses so the output would have to break out a /24 into 256 individual IPs and etc. for other CIDRs in the output. Thanks in advance!

Ask Questions Get Answers Join the Live Community
Version history
Revision #:
11 of 11
Last update:
2 weeks ago
Updated by:
 
Contributors