EDL file problem

[paul_w]

Hello -

I have created an EDL in PANOS 8.0.0 using a feed from Minemeld 0.9.40, when I commit I receive the following message:

EDL(vsys1/Skype-IPv4 ip) Downloaded file is not a text file.

Does anyone know how to correct the error ?

Thanks

[lmori]

Hi @paul_w,

you should check ms.log for additional details. Most probably this is due to a known bug in PAN-OS 8.0.0 that was fixed in the subsequent releases. The bug was related to certificate verification.

[paul_w]

Hi Imori -

I will check the log.

For information I used the process on https://gist.github.com/jtschichold/f0977e5c1ec09b3ec7d66bf80687d9da to generate the certificate.

[paul_w]

Hi @lmori

The file appears to be downloading:

2017-07-12 13:30:50.287 +0100 EDL entry(0x10a6a000, 0x1cff6000, (nil) vsys1/Skype-IPv4, 1, 1 ip) Downloaded EDL file size(1210)

There is nothing obvious to me in ms.log, I have attached some extracts for reference.

Thanks

[lmori]

Hi @paul_w,

have you tried connecting to the configured URL with the browser ? Do you see the contents ? Is there a proxy between the firewall and the EDL ?

[paul_w]

Hi @lmori -

I have connected to the URL with a browser and I can see the IP addresses listed, there are no proxies involved, the 'Test Source URL' on the EDL object gives result message 'Source URL is accessible.'

[lmori]

Hi @paul_w,

could you check MineMeld API logs for the requests of the firewall ? /opt/minemeld/logs/minemeld-web.log (or download from SYSTEM > DASHBOARD > API > LOGS)

[paul_w]

Hi @lmori - I can't find find the ip address of the firewall in the logs and there don't appear to be any obvious errors.

Logs attached.

Thanks.

[lmori]

If the firewall IP is not in the minemeld log, it means that MineMeld does not receive the EDL request from the firewall.

Could you double check that the URL in PAN-OS is correct (don't trust "Test Source URL") ?

Is there something in the middle between the firewall management interface and MineMeld that could block the session ?

[paul_w]

The EDL is on one end of an IPSEC VPN the peer traffic logs attached appear to show successful connections to the Minemeld server.

The URL shows IP addresses, extract below:

104.208.152.137-104.208.152.137
104.208.28.54-104.208.28.54
104.208.31.113-104.208.31.113
104.209.188.207-104.209.188.207
104.210.1.218-104.210.1.218
104.210.80.193-104.210.80.193
104.210.9.95-104.210.9.95
104.211.162.59-104.211.162.59
104.211.165.113-104.211.165.113
104.211.165.216-104.211.165.216
104.40.189.177-104.40.189.177
104.40.75.8-104.40.75.8
104.40.76.196-104.40.76.196
104.40.82.150-104.40.82.150
104.40.91.215-104.40.91.215
104.41.151.83-104.41.151.83
104.41.207.112-104.41.207.112
104.41.208.54-104.41.208.54
104.41.210.140-104.41.210.140
104.42.228.150-104.42.228.150

The only thing I can think of that would block the session is the peer firewall and as I say the logs appear to show a valid connection.

The system log on the EDL firewall also appears to show that the file is being downloaded and processed...or are these spurious messages ?  

[lmori]

Hi @paul_w,

the firewall is downloading 1210 bytes but the file does not contain any valid indicator.

Is the feed the O365 Skype IPv4 addresses ? That should be around 8K (306 indicators).

Could the firewall be hitting an error response page somewhere ?

Is authentication enabled on MineMeld ?

[paul_w]

Hello @lmori

Yes the feed the O365 Skype IPv4 addresses.

The only thing between the EDL and MineMeld server is the PA-5020 that is the peer for the VPN.

No authentication for output feeds is disabled

I have copy/pasted the output from the MineMeld feed to a text file and put it on a web server in another part of the network, created another EDL on the same firewall and the EDL wont populate from the web server either.

Could the problem be related in someway to the format of the data ?

For info - I have got an EBL and an EDL accessing the same MineMeld feed locally with no problems but they are running PANOS 6.1.10 and 7.1.6

Thanks.

[lmori]

Could it be that the session is triggering a URL policy deny or a captive portal on the firewall on the otehr side of the VPN ?

The EDL downloader on PAN-OS then would be downloading some data (the URL access deny page or the captive portal) and that would explain the error.

[paul_w]

Hello @lmori -

I am currently waiting for someone to test connectivity to the EDL feed from the other side of the VPN.

Looking at my problem from a different angle, do you know whether there are any external feeds available that I can try to use until I resolve my problem please ?

Thank you. 

[lmori]

You can check https://panwdbl.appspot.com - my old web application. Is way less powerful than MineMeld and contains a small subset of the feeds, but it could be enough for testing the EDLs.