Consuming mind meld feeds on Firewall

[DMurrayMCS]

Hi,

I have minemeld running on Azure and it processes and creates feeds as I would expect and can view them in a browser. The only change from the inital Azure build I have done is to install my own go-daddy SSL cert so out the box browsers will trust minemeld.

My lab has a PA-220 running 8.0.2 and when I add an external dynamic list it errors when I attempt to test it with "URL access error" BUT I can copy and paste the URL into a browser and it opens as expected.

Any idea's or hints be great !

[lmori]

HI @DMurrayMCS,

couple of questions:

- did you enable authentication on the feeds ?

- did you configure a Certificate profile for the feed ?

Thanks,

luigi

[DMurrayMCS]

Authentication - No.

Certificate profile - No and I suspect this is what is wrong ?

BTW the feed is here if you want to test it; its a summary of all O365 URL's

https://minemeld.murraycs.co.uk/feeds/MS_O365ANY

[DMurrayMCS]

OK so imported the certs and the feed now tests out ok, but when I look at the contents of the list its empty, but If I open the feed in a browser its all present ?

Drew.

[lmori · Accepted solution]

Hi @DMurrayMCS,

you should upload this into PAN-OS and use it inside a certificate profile: https://certs.godaddy.com/repository/gd-class2-root.crt (GoDaddy Class 2 Root CA)

Also remember to add "v=panousrl" in the EDL URL: https://minemeld.murraycs.co.uk/feeds/MS_O365ANY?v=panosurl

Note that to be able to see the list content in the WebUI you should use the EDL inside a policy or inside a used URL Filtering profile. If you don't use the EDL in the config in any way PAN-OS won't pull the list and the contents won't show up in the UI.

[DMurrayMCS]

All working, thank you very much for your help 🙂

[DMurrayMCS]

Totally strange but the SAME config for a dynamic list, with the SAME cert does not work on my Lab 220.

It complains that they are no valid URL's in the file - its the same feed thats working on my production 5050 ????

Are there any more logs on the 220 I can look at to work out whats going on ?

Drew.

[lmori]

Hi @DMurrayMCS,

you can check ms.log ("less mp-log ms.log" from the CLI).

Which PAN-OS version are you running on your 220 ?

[DMurrayMCS]

Im on 8.0.2 on the 220 with latest dynamic updates applied.

Log shows :-

2017-06-06 19:56:58.444 +0100 EDLRefresh job started processing. Dequeue time=2017/06/06 19:56:58 2017-06-06 19:57:00.205 +0100 client dagger reported op c
ommand was SUCCESSFUL
2017-06-06 19:57:02.213 +0100 client authd reported op command was SUCCESSFUL
2017-06-06 19:57:11.418 +0100 client dagger reported op command was SUCCESSFUL
2017-06-06 19:57:52.753 +0100 client authd reported op command was SUCCESSFUL
2017-06-06 19:57:56.119 +0100 EDLRefresh job started processing. Dequeue time=2017/06/06 19:57:56 2017-06-06 19:57:57.207 +0100 Error: pan_get_ssl_conn_fa
il_on_cert(pan_sysd_util.c:104): failed to fetch: NO_MATCHES
2017-06-06 19:57:59.043 +0100 client dagger reported op command was SUCCESSFUL
2017-06-06 19:58:00.269 +0100 Error: ebl_fetch_url_from_remote_libcurl(pan_cfg_ebl.c:1779): curl_easy_perform failed, Err(7):Couldn't connect to server
2017-06-06 19:58:00.270 +0100 EDL entry(0x10a7a000, 0x30850800, 0x2f8c1600 vsys1/O365List, 1, 1 url) calling /bin/sed -e 's/^M$//g' /opt/pancfg/mgmt/devic
es/localhost.localdomain/vsys1_O365List.ubl.tmpxx 2>/dev/null > /opt/pancfg/mgmt/devices/localhost.localdomain/vsys1_O365List.ubl.tmp
2017-06-06 19:58:00.526 +0100 Error: ebl_verify_fetched_copy(pan_cfg_ebl.c:2278): EDL entry(0x10a7a000, 0x30850800, 0x2f8c1600 vsys1/O365List, 1, 1 url) N
o valid entries found. Couldn't connect to server
2017-06-06 19:58:00.804 +0100 client authd reported op command was SUCCESSFUL
2017-06-06 19:58:01.205 +0100 EDL entry(0x10a7a000, 0x30850800, 0x2f8c1600 vsys1/O365List, 1, 1 url) Valid entries(0) lines skipped(1)
2017-06-06 19:58:01.410 +0100 EDL entry(0x10a7a000, 0x30850800, 0x2f8c1600 vsys1/O365List, 1, 1 url) No valid urls found in list file

and again 

2017-06-06 20:00:27.320 +0100 EDLRefresh job started processing. Dequeue time=2017/06/06 20:00:27 2017-06-06 20:00:30.152 +0100 Error: pan_get_ssl_conn_fa
il_on_cert(pan_sysd_util.c:104): failed to fetch: NO_MATCHES
2017-06-06 20:00:33.219 +0100 Error: ebl_fetch_url_from_remote_libcurl(pan_cfg_ebl.c:1779): curl_easy_perform failed, Err(7):Couldn't connect to server
2017-06-06 20:00:33.220 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) calling /bin/sed -e 's/^M$//g' /opt/pancfg/mgmt/devic
es/localhost.localdomain/vsys1_O365List.ubl.tmpxx 2>/dev/null > /opt/pancfg/mgmt/devices/localhost.localdomain/vsys1_O365List.ubl.tmp
2017-06-06 20:00:33.677 +0100 Error: ebl_verify_fetched_copy(pan_cfg_ebl.c:2278): EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) N
o valid entries found. Couldn't connect to server
2017-06-06 20:00:34.872 +0100 Error: ebl_update_local_file(pan_cfg_ebl.c:2717): EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) Una
ble to fetch external dynamic list. Couldn't connect to server. Using old copy for refresh.
2017-06-06 20:00:34.873 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) No changes to list file
2017-06-06 20:00:34.873 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) Remote fetch is done by worker thread 8
2017-06-06 20:00:34.873 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) Valid entries(0) lines skipped(1)
2017-06-06 20:00:35.616 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x1b2e7200 vsys1/O365List, 1, 1 url) Hourly schedule timer expires(Tue Jun 6 21:00:35 2017
)
2017-06-06 20:00:59.572 +0100 API Key is not set in cryptod
rm: cannot remove `/opt/pancfg/mgmt/wildfire-images/tmp': Is a directory
'cfg.fail-conn-on-cert': NO_MATCHES
2017-06-06 20:01:01.978 +0100 Error: pan_ebl_system_ebl_refresh_handler(pan_cfg_ebl.c:6522): EDL URL access error
2017-06-06 20:01:11.719 +0100 Error: pan_ebl_system_ebl_show_handler(pan_cfg_ebl.c:7245): EDL No valid entries
2017-06-06 20:01:20.177 +0100 Error: pan_cert_modify_node(pan_cert_ops.c:1737): Unable to extract common name
2017-06-06 20:01:20.463 +0100 client sslmgr reported op command was SUCCESSFUL
2017-06-06 20:01:22.600 +0100 Error: pan_cert_modify_node(pan_cert_ops.c:1737): Unable to extract common name
2017-06-06 20:01:22.883 +0100 client sslmgr reported op command was SUCCESSFUL

[DMurrayMCS · Accepted solution]

Thing is I can browse through firewall and read feeds fine 😕

Can't work out where next to look !

[DMurrayMCS]

OK I worked it out, kind of silly really.

My LAB is different to work, it was the service route configuration !!

Thanks for the swift reply !

Drew.