topic EDL file problem in General Topics

EDL file problem

paul_w — Wed, 12 Jul 2017 13:07:03 GMT

Hello -

I have created an EDL in PANOS 8.0.0 using a feed from Minemeld 0.9.40, when I commit I receive the following message:

EDL(vsys1/Skype-IPv4 ip) Downloaded file is not a text file.

Does anyone know how to correct the error ?

Thanks

Re: EDL file problem

lmori — Wed, 12 Jul 2017 13:23:03 GMT

Hi @paul_w,

you should check ms.log for additional details. Most probably this is due to a known bug in PAN-OS 8.0.0 that was fixed in the subsequent releases. The bug was related to certificate verification.

Re: EDL file problem

paul_w — Wed, 12 Jul 2017 13:27:53 GMT

Hi Imori -

I will check the log.

For information I used the process on https://gist.github.com/jtschichold/f0977e5c1ec09b3ec7d66bf80687d9da to generate the certificate.

Re: EDL file problem

paul_w — Wed, 12 Jul 2017 14:04:03 GMT

Hi @lmori

The file appears to be downloading:

2017-07-12 13:30:50.287 +0100 EDL entry(0x10a6a000, 0x1cff6000, (nil) vsys1/Skype-IPv4, 1, 1 ip) Downloaded EDL file size(1210)

There is nothing obvious to me in ms.log, I have attached some extracts for reference.

Thanks

Re: EDL file problem

lmori — Wed, 12 Jul 2017 14:13:27 GMT

Hi @paul_w,

have you tried connecting to the configured URL with the browser ? Do you see the contents ? Is there a proxy between the firewall and the EDL ?

Re: EDL file problem

paul_w — Wed, 12 Jul 2017 14:20:46 GMT

Hi @lmori -

I have connected to the URL with a browser and I can see the IP addresses listed, there are no proxies involved, the 'Test Source URL' on the EDL object gives result message 'Source URL is accessible.'

Re: EDL file problem

lmori — Thu, 13 Jul 2017 06:54:09 GMT

Hi @paul_w,

could you check MineMeld API logs for the requests of the firewall ? /opt/minemeld/logs/minemeld-web.log (or download from SYSTEM > DASHBOARD > API > LOGS)

Re: EDL file problem

paul_w — Thu, 13 Jul 2017 07:38:21 GMT

Hi @lmori - I can't find find the ip address of the firewall in the logs and there don't appear to be any obvious errors.

Logs attached.

Thanks.

Re: EDL file problem

lmori — Thu, 13 Jul 2017 07:43:22 GMT

If the firewall IP is not in the minemeld log, it means that MineMeld does not receive the EDL request from the firewall.

Could you double check that the URL in PAN-OS is correct (don't trust "Test Source URL") ?

Is there something in the middle between the firewall management interface and MineMeld that could block the session ?

Re: EDL file problem

paul_w — Thu, 13 Jul 2017 08:26:27 GMT

The EDL is on one end of an IPSEC VPN the peer traffic logs attached appear to show successful connections to the Minemeld server.

The URL shows IP addresses, extract below:

104.208.152.137-104.208.152.137
104.208.28.54-104.208.28.54
104.208.31.113-104.208.31.113
104.209.188.207-104.209.188.207
104.210.1.218-104.210.1.218
104.210.80.193-104.210.80.193
104.210.9.95-104.210.9.95
104.211.162.59-104.211.162.59
104.211.165.113-104.211.165.113
104.211.165.216-104.211.165.216
104.40.189.177-104.40.189.177
104.40.75.8-104.40.75.8
104.40.76.196-104.40.76.196
104.40.82.150-104.40.82.150
104.40.91.215-104.40.91.215
104.41.151.83-104.41.151.83
104.41.207.112-104.41.207.112
104.41.208.54-104.41.208.54
104.41.210.140-104.41.210.140
104.42.228.150-104.42.228.150

The only thing I can think of that would block the session is the peer firewall and as I say the logs appear to show a valid connection.

The system log on the EDL firewall also appears to show that the file is being downloaded and processed...or are these spurious messages ?

Re: EDL file problem

lmori — Fri, 14 Jul 2017 09:05:36 GMT

Hi @paul_w,

the firewall is downloading 1210 bytes but the file does not contain any valid indicator.

Is the feed the O365 Skype IPv4 addresses ? That should be around 8K (306 indicators).

Could the firewall be hitting an error response page somewhere ?

Is authentication enabled on MineMeld ?

Re: EDL file problem

paul_w — Fri, 14 Jul 2017 09:51:49 GMT

Hello @lmori

Yes the feed the O365 Skype IPv4 addresses.

The only thing between the EDL and MineMeld server is the PA-5020 that is the peer for the VPN.

No authentication for output feeds is disabled

I have copy/pasted the output from the MineMeld feed to a text file and put it on a web server in another part of the network, created another EDL on the same firewall and the EDL wont populate from the web server either.

Could the problem be related in someway to the format of the data ?

For info - I have got an EBL and an EDL accessing the same MineMeld feed locally with no problems but they are running PANOS 6.1.10 and 7.1.6

Thanks.

Re: EDL file problem

lmori — Fri, 14 Jul 2017 14:12:49 GMT

Could it be that the session is triggering a URL policy deny or a captive portal on the firewall on the otehr side of the VPN ?

The EDL downloader on PAN-OS then would be downloading some data (the URL access deny page or the captive portal) and that would explain the error.

Re: EDL file problem

paul_w — Mon, 17 Jul 2017 08:44:08 GMT

Hello @lmori -

I am currently waiting for someone to test connectivity to the EDL feed from the other side of the VPN.

Looking at my problem from a different angle, do you know whether there are any external feeds available that I can try to use until I resolve my problem please ?

Thank you.

Re: EDL file problem

lmori — Tue, 18 Jul 2017 11:01:15 GMT

You can check https://panwdbl.appspot.com - my old web application. Is way less powerful than MineMeld and contains a small subset of the feeds, but it could be enough for testing the EDLs.

Re: EDL file problem

paul_w — Thu, 20 Jul 2017 08:57:08 GMT

Hello @lmori -

I have had confirmation that a device on the same network as the firewall management interface can access the feed via a browser and can see the IP list without any block pages being displayed.

Are there any test commands that can be run from the firewall, is it possible to see the contents of the file that is being downloaded ?

Thanks.

Re: EDL file problem

paul_w — Tue, 25 Jul 2017 10:52:50 GMT

Hello @lmori

I have been able to populate the EDL object from other web servers so I believe I must have encountered the bug you referred to in your first reply...even though I was unable to find any error messages in the log file.

Thank you for your assistance.

Kind Regards, paul_w

Re: EDL file problem

RbadigerCY — Thu, 28 Feb 2019 19:29:49 GMT

Hi All,

I am facing a strange EDL issue on my firewalls. I have various models of PAN and they all are on 8.0.7+ versions.

What happens?

- EDL works fine without any issues, suddenly EDL stops working. Access to all IP addresses inside that specific EDL breaks.

- I checked the EDL list on CLI with following command, and I see all entries listed under the EDL:

requesst system external-list show type ip name XXXXX

Workaround - I manually refresh the EDL. And once successfully completed, everything starts working.

Suspected issue -

PAN-100244 - Seen on PAN-OS 8.1.4, and fixed on PAN-OS 8.1.5.

I am noticing this issue on PAN-OS 8.0.7 as well, is this something related to OS issue? Is there any permanent fix for this issue? Please advise.

Thanks in advance.

Regards,

Raghav

Re: EDL file problem

lmori — Tue, 12 Mar 2019 17:03:38 GMT

Hi @RbadigerCY,

this seems something related to PAN-OS. Do you have a ticket for this?

Luigi

Re: EDL file problem

RbadigerCY — Wed, 13 Mar 2019 10:22:46 GMT

Yes, this is a bug related to PAn-OS 8.1.4, which is okay for me. However I have other PAN devices whihc are running on other OS, they are also having same issues.

Regards,

Raghav