EDL file empty?

[irt-unimi]

We installed Minemeld on Ubuntu 14.04 as documented and it's mostly working, except that from time to time the output lists are empty and PAN-OS Monitor>System complains:

medium::EDL(DSHIELD20) Downloaded file is either not a text file or empty file. Using old copy for refresh.

It's an unpredictable behavior and the EDL file comes back after a while.

I can see in /opt/minemeld/log/minemeld-web.log that the file size is sometimes 1 instead of, say, 560 (DShield20) or 22419 (Spamhaus DROP) so it's definitely a server issue.

What's going on?

Jan

[lmori]

@irt-unimi, could you check minemeld-engine.log for errors in accessing dshield ?

luigi

[irt-unimi]

Hi Luigi,

minemeld-engine.log does indeed show something interesting in the timeframe of the outage:

2017-01-27T07:11:39 (22615)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org
2017-01-27T07:11:40 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497499987 sudden_death
2017-01-27T07:11:40 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497499987 age_out
2017-01-27T07:11:40 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497499987 gc
2017-01-27T07:11:55 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497515622 age_out
2017-01-27T07:16:12 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497772715 age_out
2017-01-27T07:20:29 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485498029750 age_out
2017-01-27T07:21:59 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485498119031 poll

Do you think it's DSHIELD not responding? And what can be done?

[lmori]

Hi @irt-unimi,

logs look normal. Please could you share a screenshot of the STATS section of the dshield.block Miner ?

[irt-unimi]

Here is the DSHIELD stats, last outage was Jan 27th, and the Spamhaus EDROP (last outage Jan 26th) seems that the stats are only for last 24hrs.

[lmori]

hi @irt-unimi,

counters look good. Would you mind sending your minemeld-web.log file over at lmori@paloaltonetworks.com ?

Thanks,

luigi

[lmori · Accepted solution]

Hi @irt-unimi,

found the problem, it is already fixed in the current beta. There will be a new release by the end of this week with the fix included. Let me know if you would like to install the beta.

luigi

[irt-unimi]

Great Luigi, thank you.

[tkirk]

Hi Luigi,

Did the fix make 8.0 GA, as I'm running into the same problem?

The MM lists are accessible via the browser, but I get and error from PAN-OS that they are not a text file. I have disabled the certificate profile, tried URLs, IP ranges & individuals (statics), but all display the same error. The MM engine log seems to be fine.

Bouced MM and firewall..

Thanks,

Tim

[lmori]

Hi @tkirk,

could you check the ms.log file on PAN-OS for additional details ?

> tail mp-log ms.log