topic Re: Hail-a-taxii PhishTank Taxii Issues in General Topics

Hail-a-taxii PhishTank Taxii Issues

vedd3r — Fri, 06 Oct 2017 02:20:13 GMT

Hi,

Just wanted to ask whether the hail-a-taxii miner for Phishtank is broken or not. What I'm experiencing is that, on first creation it pulls the data for the first time, and then nothing from there on - no new data being pulled. I ran a pull on hail-a-taxii PhishTank taxii feed using Anomali STAXX and confirm there were new URLs provided each day - just that it's not being pulled by MineMeld. I'm seeing in all of my MineMeld installations (test env and production env).

Below is screen shot of my test system and the first pull (Oct 2nd) and nothing newer than that comes out.

From the log, only the first pull was captured and nothing after that.Today's pull was a success but didn't yield any new data and logs doesn't show them either.

Re: Hail-a-taxii PhishTank Taxii Issues

lmori — Fri, 06 Oct 2017 09:21:33 GMT

Hi @vedd3r,

thanks for reporting this. I will check this (I need some days to wait for the updates) and let you know.

luigi

Re: Hail-a-taxii PhishTank Taxii Issues

vedd3r — Mon, 09 Oct 2017 06:22:34 GMT

Thanks @lmori! I'm activating the rest of Hail-a-taxii prototypes as well and monitoring the output.

Re: Hail-a-taxii PhishTank Taxii Issues

vedd3r — Fri, 13 Oct 2017 02:36:23 GMT

I've checked on the status for all the Hail-A-Taxii (HAT) feeds and it seemed it's not limited to PhishTank only. None of the HAT prototypes are pulling new data.

Re: Hail-a-taxii PhishTank Taxii Issues

lmori — Mon, 16 Oct 2017 04:17:11 GMT

Hi @vedd3r,

my phishtank miner is pulling updates, may I know the MM release you are running on ?

Thanks,

luigi

Re: Hail-a-taxii PhishTank Taxii Issues

vedd3r — Mon, 16 Oct 2017 06:51:42 GMT

Hi @lmori

I'm using 0.9.42 for all my installations. All of them are not pulling the updates after the initial pull. I've also asked a couple of my friends who uses MM and they confirmed they're experiencing the same issue as well.

Re: Hail-a-taxii PhishTank Taxii Issues

lmori — Mon, 16 Oct 2017 07:23:28 GMT

Hi @vedd3r,

thanks for the feedback. I will check everything again as soon as hailataxii comes back online (it's returning 500s now) and let you know about my findings.

luigi

Re: Hail-a-taxii PhishTank Taxii Issues

lmori — Thu, 26 Oct 2017 14:26:07 GMT

Hi @vedd3r,

I have monitored phishtank miner for some days and I have seen updates. This is my graph:

This is the number of indicators over time in the graph, green bars below are the updates generated by the Miner:

Would you mind sending me the minemeld-engine.log ? I am reachable at lmori@paloaltonetworks.com.

Thanks!

luigi

Re: Hail-a-taxii PhishTank Taxii Issues

vedd3r — Mon, 30 Oct 2017 06:15:02 GMT

Hi @lmori

I've sent an email to you with the logs. There was a power trip on 25th Oct which caused the vm host and Minemeld guest vm to restart - 95 new indicators were pulled by the PhishTank prototype on that day and after that it went silent again as if nothing new is there.

Let me know if you need additional logs/data.

Thanks.

Re: Hail-a-taxii PhishTank Taxii Issues

lmori — Thu, 09 Nov 2017 09:03:00 GMT

Hi @vedd3r,

thaks, I am pretty sure we have isolated the issue - if all the tests goes as expected we will release an HF for this.

Thanks !

Luigi