Help on CSV Output Feed
Showing results for 
Search instead for 
Did you mean: 

Help on CSV Output Feed

L0 Member

I'm looking to output feeds to a format that I can ingest in some log analysis tools, and need to output fields that I have defined in miners. Is there any information on how to access that data and output it?


L7 Applicator

Could give us more details about the format and fields your log analsys platform could ingest ?

Sure. Thanks for the quick response.


So we have some custom scripts in place that parse any number of feeds, normalize the data, and feed it into our log analysis solution. For each feed we extract date, source_name, ioc_desc, ioc. We then add category, ioc_type, confidence. I'm looking to replace soem of our custom scripts with this, but I'm having trouble getting the data back out in that format. 


I can do teh parsing just fine with teh miners, filter/aggregate with the processors, but I can't get the output to limit the fields and send it in tsv/csv/syslog format as is. Poking around in the flask part of the code I was able to tweak the to output in tsv  - just not sure if thats the best way to do it. Also a little stuck on how to specify field filters in teh output. 



Hi kx1499, could be the best place to do this. I think the best way to restrict field is sending the required fields as a URL parameter. Do you have an example of the tsv output format required by your log analysis tool ? Is the log analysis tool a commercial solution ?




Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!