topic Re: EDL file empty? in General Topics

EDL file empty?

irt-unimi — Thu, 26 Jan 2017 16:16:09 GMT

We installed Minemeld on Ubuntu 14.04 as documented and it's mostly working, except that from time to time the output lists are empty and PAN-OS Monitor>System complains:

medium::EDL(DSHIELD20) Downloaded file is either not a text file or empty file. Using old copy for refresh.

It's an unpredictable behavior and the EDL file comes back after a while.

I can see in /opt/minemeld/log/minemeld-web.log that the file size is sometimes 1 instead of, say, 560 (DShield20) or 22419 (Spamhaus DROP) so it's definitely a server issue.

What's going on?

Jan

Re: EDL file empty?

lmori — Mon, 30 Jan 2017 14:27:25 GMT

@irt-unimi, could you check minemeld-engine.log for errors in accessing dshield ?

luigi

Re: EDL file empty?

irt-unimi — Mon, 30 Jan 2017 14:49:02 GMT

Hi Luigi,

minemeld-engine.log does indeed show something interesting in the timeframe of the outage:

2017-01-27T07:11:39 (22615)connectionpool._new_conn INFO: Starting new HTTPS connection (1): www.dshield.org
2017-01-27T07:11:40 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497499987 sudden_death
2017-01-27T07:11:40 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497499987 age_out
2017-01-27T07:11:40 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497499987 gc
2017-01-27T07:11:55 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497515622 age_out
2017-01-27T07:16:12 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485497772715 age_out
2017-01-27T07:20:29 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485498029750 age_out
2017-01-27T07:21:59 (22615)basepoller._actor_loop INFO: DSHIELD20BLOCKLIST - command: 1485498119031 poll

Do you think it's DSHIELD not responding? And what can be done?

Re: EDL file empty?

lmori — Mon, 30 Jan 2017 14:50:48 GMT

Hi @irt-unimi,

logs look normal. Please could you share a screenshot of the STATS section of the dshield.block Miner ?

Re: EDL file empty?

irt-unimi — Mon, 30 Jan 2017 15:02:44 GMT

Here is the DSHIELD stats, last outage was Jan 27th, and the Spamhaus EDROP (last outage Jan 26th) seems that the stats are only for last 24hrs.

Re: EDL file empty?

lmori — Mon, 30 Jan 2017 15:05:33 GMT

hi @irt-unimi,

counters look good. Would you mind sending your minemeld-web.log file over at lmori@paloaltonetworks.com ?

Thanks,

luigi

Re: EDL file empty?

lmori — Tue, 31 Jan 2017 08:33:35 GMT

Hi @irt-unimi,

found the problem, it is already fixed in the current beta. There will be a new release by the end of this week with the fix included. Let me know if you would like to install the beta.

luigi

Re: EDL file empty?

irt-unimi — Tue, 31 Jan 2017 09:28:30 GMT

Great Luigi, thank you.

Re: EDL file empty?

tkirk — Tue, 14 Feb 2017 09:38:28 GMT

Hi Luigi,

Did the fix make 8.0 GA, as I'm running into the same problem?

The MM lists are accessible via the browser, but I get and error from PAN-OS that they are not a text file. I have disabled the certificate profile, tried URLs, IP ranges & individuals (statics), but all display the same error. The MM engine log seems to be fine.

Bouced MM and firewall..

Thanks,

Tim

Re: EDL file empty?

lmori — Wed, 15 Feb 2017 10:44:41 GMT

Hi @tkirk,

could you check the ms.log file on PAN-OS for additional details ?

> tail mp-log ms.log