Hail-a-taxii PhishTank Taxii Issues

[vedd3r]

Hi,

Just wanted to ask whether the hail-a-taxii miner for Phishtank is broken or not. What I'm experiencing is that, on first creation it pulls the data for the first time, and then nothing from there on - no new data being pulled. I ran a pull on hail-a-taxii PhishTank taxii feed using Anomali STAXX and confirm there were new URLs provided each day - just that it's not being pulled by MineMeld. I'm seeing in all of my MineMeld installations (test env and production env).

Below is screen shot of my test system and the first pull (Oct 2nd) and nothing newer than that comes out.

From the log, only the first pull was captured and nothing after that.Today's pull was a success but didn't yield any new data and logs doesn't show them either.

[lmori]

Hi @vedd3r,

thanks for reporting this. I will check this (I need some days to wait for the updates) and let you know.

luigi

[vedd3r]

Thanks @lmori! I'm activating the rest of Hail-a-taxii prototypes as well and monitoring the output.

[vedd3r]

I've checked on the status for all the Hail-A-Taxii (HAT) feeds and it seemed it's not limited to PhishTank only. None of the HAT prototypes are pulling new data.

[lmori]

Hi @vedd3r,

my phishtank miner is pulling updates, may I know the MM release you are running on ?

Thanks,

luigi

[vedd3r]

Hi @lmori

I'm using 0.9.42 for all my installations. All of them are not pulling the updates after the initial pull. I've also asked a couple of my friends who uses MM and they confirmed they're experiencing the same issue as well.

[lmori]

Hi @vedd3r,

thanks for the feedback. I will check everything again as soon as hailataxii comes back online (it's returning 500s now) and let you know about my findings.

luigi

[lmori]

Hi @vedd3r,

I have monitored phishtank miner for some days and I have seen updates. This is my graph:

This is the number of indicators over time in the graph, green bars below are the updates generated by the Miner:

Would you mind sending me the minemeld-engine.log ? I am reachable at lmori@paloaltonetworks.com.

Thanks!

luigi

[vedd3r]

Hi @lmori

I've sent an email to you with the logs. There was a power trip on 25th Oct which caused the vm host and Minemeld guest vm to restart - 95 new indicators were pulled by the PhishTank prototype on that day and after that it went silent again as if nothing new is there.

Let me know if you need additional logs/data.

Thanks.

[lmori]

Hi @vedd3r,

thaks, I am pretty sure we have isolated the issue - if all the tests goes as expected we will release an HF for this.

Thanks !

Luigi