Consuming mind meld feeds on Firewall

L1 Bithead

Consuming mind meld feeds on Firewall

Hi,

 

I have minemeld running on Azure and it processes and creates feeds as I would expect and can view them in a browser. The only change from the inital Azure build I have done is to install my own go-daddy SSL cert so out the box browsers will trust minemeld.

 

My lab has a PA-220 running 8.0.2 and when I add an external dynamic list it errors when I attempt to test it with "URL access error" BUT I can copy and paste the URL into a browser and it opens as expected.

 

Any idea's or hints be great !

L7 Applicator

Re: Consuming mind meld feeds on Firewall

HI @DMurrayMCS,

couple of questions:

- did you enable authentication on the feeds ?

- did you configure a Certificate profile for the feed ?

 

Thanks,

luigi

L1 Bithead

Re: Consuming mind meld feeds on Firewall

Authentication - No.

 

Certificate profile - No and I suspect this is what is wrong ?

 

BTW the feed is here if you want to test it; its a summary of all O365 URL's

 

https://minemeld.murraycs.co.uk/feeds/MS_O365ANY

 

 

L1 Bithead

Re: Consuming mind meld feeds on Firewall

OK so imported the certs and the feed now tests out ok, but when I look at the contents of the list its empty, but If I open the feed in a browser its all present ?

 

Drew.

L7 Applicator

Re: Consuming mind meld feeds on Firewall

Hi @DMurrayMCS,

you should upload this into PAN-OS and use it inside a certificate profile: https://certs.godaddy.com/repository/gd-class2-root.crt (GoDaddy Class 2 Root CA)

 

Also remember to add "v=panousrl" in the EDL URL: https://minemeld.murraycs.co.uk/feeds/MS_O365ANY?v=panosurl

 

Note that to be able to see the list content in the WebUI you should use the EDL inside a policy or inside a used URL Filtering profile. If you don't use the EDL in the config in any way PAN-OS won't pull the list and the contents won't show up in the UI.

L1 Bithead

Re: Consuming mind meld feeds on Firewall

All working, thank you very much for your help :-)

L1 Bithead

Re: Consuming mind meld feeds on Firewall

Totally strange but the SAME config for a dynamic list, with the SAME cert does not work on my Lab 220.

It complains that they are no valid URL's in the file - its the same feed thats working on my production 5050 ????

 

Are there any more logs on the 220 I can look at to work out whats going on ?

 

Drew.

L7 Applicator

Re: Consuming mind meld feeds on Firewall

Hi @DMurrayMCS,

you can check ms.log ("less mp-log ms.log" from the CLI).

Which PAN-OS version are you running on your 220 ?

L1 Bithead

Re: Consuming mind meld feeds on Firewall

Im on 8.0.2 on the 220 with latest dynamic updates applied.

 

Log shows :-

 

2017-06-06 19:56:58.444 +0100 EDLRefresh job started processing. Dequeue time=2017/06/06 19:56:58 2017-06-06 19:57:00.205 +0100 client dagger reported op c
ommand was SUCCESSFUL
2017-06-06 19:57:02.213 +0100 client authd reported op command was SUCCESSFUL
2017-06-06 19:57:11.418 +0100 client dagger reported op command was SUCCESSFUL
2017-06-06 19:57:52.753 +0100 client authd reported op command was SUCCESSFUL
2017-06-06 19:57:56.119 +0100 EDLRefresh job started processing. Dequeue time=2017/06/06 19:57:56 2017-06-06 19:57:57.207 +0100 Error: pan_get_ssl_conn_fa
il_on_cert(pan_sysd_util.c:104): failed to fetch: NO_MATCHES
2017-06-06 19:57:59.043 +0100 client dagger reported op command was SUCCESSFUL
2017-06-06 19:58:00.269 +0100 Error: ebl_fetch_url_from_remote_libcurl(pan_cfg_ebl.c:1779): curl_easy_perform failed, Err(7):Couldn't connect to server
2017-06-06 19:58:00.270 +0100 EDL entry(0x10a7a000, 0x30850800, 0x2f8c1600 vsys1/O365List, 1, 1 url) calling /bin/sed -e 's/^M$//g' /opt/pancfg/mgmt/devic
es/localhost.localdomain/vsys1_O365List.ubl.tmpxx 2>/dev/null > /opt/pancfg/mgmt/devices/localhost.localdomain/vsys1_O365List.ubl.tmp
2017-06-06 19:58:00.526 +0100 Error: ebl_verify_fetched_copy(pan_cfg_ebl.c:2278): EDL entry(0x10a7a000, 0x30850800, 0x2f8c1600 vsys1/O365List, 1, 1 url) N
o valid entries found. Couldn't connect to server
2017-06-06 19:58:00.804 +0100 client authd reported op command was SUCCESSFUL
2017-06-06 19:58:01.205 +0100 EDL entry(0x10a7a000, 0x30850800, 0x2f8c1600 vsys1/O365List, 1, 1 url) Valid entries(0) lines skipped(1)
2017-06-06 19:58:01.410 +0100 EDL entry(0x10a7a000, 0x30850800, 0x2f8c1600 vsys1/O365List, 1, 1 url) No valid urls found in list file

 

 

and again 

 

 

2017-06-06 20:00:27.320 +0100 EDLRefresh job started processing. Dequeue time=2017/06/06 20:00:27 2017-06-06 20:00:30.152 +0100 Error: pan_get_ssl_conn_fa
il_on_cert(pan_sysd_util.c:104): failed to fetch: NO_MATCHES
2017-06-06 20:00:33.219 +0100 Error: ebl_fetch_url_from_remote_libcurl(pan_cfg_ebl.c:1779): curl_easy_perform failed, Err(7):Couldn't connect to server
2017-06-06 20:00:33.220 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) calling /bin/sed -e 's/^M$//g' /opt/pancfg/mgmt/devic
es/localhost.localdomain/vsys1_O365List.ubl.tmpxx 2>/dev/null > /opt/pancfg/mgmt/devices/localhost.localdomain/vsys1_O365List.ubl.tmp
2017-06-06 20:00:33.677 +0100 Error: ebl_verify_fetched_copy(pan_cfg_ebl.c:2278): EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) N
o valid entries found. Couldn't connect to server
2017-06-06 20:00:34.872 +0100 Error: ebl_update_local_file(pan_cfg_ebl.c:2717): EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) Una
ble to fetch external dynamic list. Couldn't connect to server. Using old copy for refresh.
2017-06-06 20:00:34.873 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) No changes to list file
2017-06-06 20:00:34.873 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) Remote fetch is done by worker thread 8
2017-06-06 20:00:34.873 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x2d7d7b00 vsys1/O365List, 1, 1 url) Valid entries(0) lines skipped(1)
2017-06-06 20:00:35.616 +0100 EDL entry(0x10a7a000, 0x20d90000, 0x1b2e7200 vsys1/O365List, 1, 1 url) Hourly schedule timer expires(Tue Jun 6 21:00:35 2017
)
2017-06-06 20:00:59.572 +0100 API Key is not set in cryptod
rm: cannot remove `/opt/pancfg/mgmt/wildfire-images/tmp': Is a directory
'cfg.fail-conn-on-cert': NO_MATCHES
2017-06-06 20:01:01.978 +0100 Error: pan_ebl_system_ebl_refresh_handler(pan_cfg_ebl.c:6522): EDL URL access error
2017-06-06 20:01:11.719 +0100 Error: pan_ebl_system_ebl_show_handler(pan_cfg_ebl.c:7245): EDL No valid entries
2017-06-06 20:01:20.177 +0100 Error: pan_cert_modify_node(pan_cert_ops.c:1737): Unable to extract common name
2017-06-06 20:01:20.463 +0100 client sslmgr reported op command was SUCCESSFUL
2017-06-06 20:01:22.600 +0100 Error: pan_cert_modify_node(pan_cert_ops.c:1737): Unable to extract common name
2017-06-06 20:01:22.883 +0100 client sslmgr reported op command was SUCCESSFUL

 

 

L1 Bithead

Re: Consuming mind meld feeds on Firewall

Thing is I can browse through firewall and read feeds fine :-/

 

Can't work out where next to look !

Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the Live Community as a whole!

The Live Community thanks you for your participation!