get this message with External EDL server

[dtran]

I have multiple firewalls that are connected to my linux EDL server to retrieve both IP address and URL list.  I am using http and not https and the firewall is scheduled to pull the data every hours from the EDL web server.

on the firewall  system logs, I see messages in "medium" catergory like this: "description contains 'EDL(EDL_Whitelist_IPs) Either EDL file was not updated at remote end or Downloaded file is not a text file. Using old copy for refresh.. inode/x-empty"

It is not failing all the times and it works like 70% of the times.  When I get this above message, I check the tcpdump between the firewall management and the web server, I can see the firewalls successfully pulls the list from the web server, and yet, I still this message.

Any ideas?

[BPry]

@dtran,

Most commonly when I've had folks run into this in a non-consistent basis it's because whatever they are using to host the EDL isn't returning a 200 OK message consistently. When you look at the packet captures you've taken are you seeing a 200 OK returned and the proper Content-Type being recorded? 

[dtran]

@BPry:  I knew you were going to ask me that.  The answer is YES.  Even when I see that message on the PAN firewalls, I get 200 OK and the proper Content-Type is being recorded.

Here is an snip of it, the capture on the management interface:

GET /Internet-cciesec2011_iplist.txt HTTP/1.1

Host: edl.cciesec2011.com

Accept: */*

HTTP/1.1 200 OK

Date: Thu, 04 Nov 2021 16:00:04 GMT

Server: Apache/2.2.15 (Red Hat)

Last-Modified: Mon, 01 Nov 2021 19:56:59 GMT

Accept-Ranges: bytes

Content-Length: 6309

Cache-Control: max-age=0, no-cache, no-store, must-revalidate

Pragma: no-cache

Note: CACHING IS DISABLED ON HOST

Expires: Wed, 11 Jan 2023 05:00:00 GMT

Connection: close

Content-Type: text/plain; charset=UTF-8

4.2.2.2/32

4.2.2.1/32

Any other ideas?

Any other ideas?

[BPry]

@dtran,

If your positive that the server is offering up the file and it's getting to the firewall (and the captures are obviously verifying that) then I would look to see if any obvious issues are being recorded in ms.log.

[dtran]

I am 100% positive that the server is offering the file and confirmed by the capture.

I opened a TAC case with PAN and they suspsect a "bug".  What else is new, right?

I am not seeing this issue with PAN-OS 8.1.17

Btw, there is another issue with 9.1.x.  Look like PAN takes away the ability for you to see whether you use http or https from the CLI.  You can see that in 8.1.x, you can see the source as http but nowhere to be found in 9.1.x.  WTF!!!

PAN-OS: 8.1.17
request system external-list show type ip name EDL_XXX_YYY

vsys1/EDL_iplist.txt:
Next update at : Mon Nov 8 16:00:19 2021
Source : http://X.X.X.X/EDL_iplist.txt
Referenced : Yes
Valid : Yes
Auth-Valid : Yes

Total valid entries : 418
Total invalid entries : 0
Valid ips:
101.80.0.0/16
101.81.0.0/16

PAN-OS: 9.1.10
request system external-list show type ip name EDL_XXX_YYY

EDL_XXX_YYY
Total valid entries : 418
Total ignored entries : 0
Total invalid entries : 0
Total displayed entries : 100
Valid ips:
101.80.0.0/16
101.81.0.0/16

[dtran]

Just as I expected, another bug from Palo Alto.  It is resolved in 9.1.11 and later version:

PAN-160253

Fixed an issue where only one medium-severity system log was generated if either the EDL file wasn't updated at the remote end or the downloaded file wasn't a text file.