Hello I am attempting to create a miner using a paid threat intelligence providers API. The data deleivered is in a text format however the URL doesn't end in .txt. The URL does require basic authentication to view the data.
I have built my new prototype based off the dsheild.block prototype.
I have some questions regarding the authentication and the indicators and transform settings.
The API URL contains data in the below format with no headers above. just a giant list of text delimited with spaces and seperated into individual lines:
5.188.10.3 #Protection IP List: "hardcoded C2 for malicious downloader" Added 2018-03-14T22:49:12Z (59.939,30.3158) RU St Petersburg, Russia
Question 1: Is the basic authentication peice something I add into the prototype?
Question 2: I removed the following portions of the original dsheild.block fields
I modified the indicator portion to only look for one IP address: regex: ^([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})
I modified the tranform to only list 1 value transform: \1
Does this look correct considering my data format?
Question 3: Their API does support a basic auth directly in the URL example: https://<api_username>:<api_password>@someurl.com/pan. I don't want to have my username and password in plain text within the prototype, how do I get around this?]
On a side note I have saveds this prototype and added the node. However, none of my indicators are being pulled. I'm sure I have screwed it up somewhere.
If you need any other information please let me know.
Thanks,
Eddie
... View more