VPN issue with a single stack for multiple firewalls

[RomainDuez]

Hello the Palo Alto community,

I'm trying to create a simple template stack for firewalls with the same topology (WAN interface on eth1/1, 1/2.x for internal VLAN, etc...) and use variables for each device.

I'm facing an issue with the VPN part. If I don't create the IKE Gateway locally on the firewalls, I get a failed commit without any further information.
For the IKE Gateway, I use 2 variables in the template. One for the local ip address (x.x.x.x/x) and one for the peer address (x.x.x.x)

Has anyone encountered this issue?

This is a lab environment so I have no problem crashing the devices...

[Claw4609 · Accepted solution]

We currently have variables setup for some of our IKE gateway config so at least to some extent this can work. Do you have your local and peer identifiers set? If you hop over to the CLI are you able to get more info on the commit failure? How to identify the commit failure reason when no error message... - Knowledge Base - Palo Alto Netw...

[RomainDuez]

Hello Claw4609,
Thanks for the feedback. 
I think I found the issue in the logs. I have the following invalid expressions
XPath error : Invalid expression
../../../../../../interface/ethernet1/1/ip/entry/@name[not(''='yes')]
XPath error : Invalid expression
../../../../../../interface/ethernet1/1/ip/entry/@name[not(''='yes')]
XPath error : Invalid expression
../../../../../../interface/ethernet1/1/ipv6/address/entry/@name[''='yes']
XPath error : Invalid expression
../../../../../../interface/ethernet1/1/ipv6/address/entry/@name[''='yes']
XPath error : Invalid expression
../../../../../../../../../../../network/interface/ethernet1/1[name()='vlan' or name()='tunnel' or name()='loopback']/ip/entry/@name
XPath error : Invalid expression
../../../../../../../../../../../network/interface/ethernet1/1[name()='vlan' or name()='tunnel' or name()='loopback']/ip/entry/@name

I'm going to look into how to solve this part...

The full logs are below:

$externalIP-GW
$localADDR
$peerADDR
2024-01-09 16:20:42.218 +0100 Error: pan_cfg_mgr_get_tpl_disabled(pan_cfg_mgr.c:8474): failed to fetch: NO_MATCHES
2024-01-09 16:20:42.581 +0100 CommitAll job started processing. Dequeue time=2024/01/09 16:20:42. JobId=106.User: Panorama-admin
2024-01-09 16:20:44.064 +0100 Panorama push template ROD-TP_STACK-DC with merge-with-candidate-cfg flags set.JobId=106.User=Panorama-admin. Dequeue time=2024/01/09 16:20:42. TPL version: 547.
2024-01-09 16:20:44.078 +0100 Error: pan_cfg_mgr_get_tpl_disabled(pan_cfg_mgr.c:8474): failed to fetch: NO_MATCHES
2024-01-09 16:20:44.080 +0100 Error: pan_cfg_transform_fullpath(pan_cfg_utils.c:6599): error generating transform /opt/pancfg/mgmt/factory/tplrenamemapfrompushreq.xsl
2024-01-09 16:20:44.080 +0100 Error: pan_cfg_tpl_renamemap_from_request(pan_cfg_templates.c:4863): failed to generate tpl rename map from request
2024-01-09 16:20:44.080 +0100 Error: pan_cfg_pushtpl_autogen_config_merge(pan_cfg_templates.c:6596): No sd-wan plugin node found under pushtpl. Proceeding with template push
2024-01-09 16:20:44.096 +0100 cfg-version 13.1.0 in pushed shared policy file /opt/pancfg/mgmt/sp/vsys1/sp-config.xml, different from my version 10.0.0, need to do transforms
2024-01-09 16:20:46.817 +0100 Error: pan_cfg_sp_generate_candidate_vsys_sps_by_root(pan_cfg_shared_policy.c:5361): no policy node under push request
2024-01-09 16:20:46.942 +0100 Error: pan_cfg_sp_generate_candidate_vsys_sps_by_root(pan_cfg_shared_policy.c:5361): no policy node under push request
2024-01-09 16:20:46.943 +0100 detail : Commit from Panorama. Merged with candidate config: Yes. Commit parameters: force=false, device_network=true, shared_object=true. Commit All Vsys.
2024-01-09 16:20:46.943 +0100 Takes 3 seconds to generate commit candidate in cfg_by_cookie.
2024-01-09 16:20:46.950 +0100 Created Verify Thread
2024-01-09 16:20:46.950 +0100 Schema validation including uuid check for job 106 takes 0 seconds
2024-01-09 16:20:47.909 +0100 Config buf size 12105115
2024-01-09 16:20:48.132 +0100 Error: _pan_merge_content_preview_pre_commit(pan_cfg_commit_handler.c:3070): Merging content preview - Could not find content preview application node.
2024-01-09 16:20:48.753 +0100 Warning: httpd_phase1(httpd_commit.c:96): No change to HTTPD config.
2024-01-09 16:20:48.753 +0100 HTTPD Commit Phase1 end. (0)
2024-01-09 16:20:48.753 +0100 HTTPD Phase1 complete.
2024-01-09 16:20:48.804 +0100 Error: pan_cfg_transform_fullpath(pan_cfg_utils.c:6599): error generating transform /opt/plugins/xsl/input-deviceconfig.xsl
/usr/local/bin/bin_scripts/pan_category_exists.sh: line 2: [: too many arguments
2024-01-09 16:20:48.967 +0100 Could not find url vendor, returning paloaltonetworks as default
2024-01-09 16:20:49.424 +0100 Error: pan_remove_double_encrypt_for_private_keys(pan_ops_common_cert.c:7276): Failed to get certificate node for shared
2024-01-09 16:20:49.432 +0100 Takes 0 seconds to find duplicate UUIDs in config.
2024-01-09 16:20:49.432 +0100 Verifying Configuration
XPath error : Invalid expression
../../../../../../interface/ethernet1/1/ip/entry/@name[not(''='yes')]
^
XPath error : Invalid expression
../../../../../../interface/ethernet1/1/ip/entry/@name[not(''='yes')]
^
XPath error : Invalid expression
../../../../../../interface/ethernet1/1/ipv6/address/entry/@name[''='yes']
^
XPath error : Invalid expression
../../../../../../interface/ethernet1/1/ipv6/address/entry/@name[''='yes']
^
XPath error : Invalid expression
../../../../../../../../../../../network/interface/ethernet1/1[name()='vlan' or name()='tunnel' or name()='loopback']/ip/entry/@name
^
XPath error : Invalid expression
../../../../../../../../../../../network/interface/ethernet1/1[name()='vlan' or name()='tunnel' or name()='loopback']/ip/entry/@name
^
2024-01-09 16:20:49.668 +0100 Takes 0 seconds to verify schema.
2024-01-09 16:20:49.668 +0100 Clearing commit completion cache2024-01-09 16:20:49.711 +0100 server hash setting for snmptrap is created
2024-01-09 16:20:49.711 +0100 server hash setting for syslog is created
2024-01-09 16:20:49.711 +0100 server hash setting for email is created
2024-01-09 16:20:49.711 +0100 server hash setting for amqp is created
2024-01-09 16:20:49.711 +0100 server hash setting for http is created
2024-01-09 16:20:49.711 +0100 Warning: pan_hash_init(pan_hash.c:113): nbuckets 5 is not power of 2!
2024-01-09 16:20:49.717 +0100 debug vsys(1)
2024-01-09 16:20:49.717 +0100 debug vsys(1)
2024-01-09 16:20:50.428 +0100 Error: pan_syslog_settings_parse(pan_server_settings.c:1840): Could not find dest entry (syslog) in hash
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.542 +0100 amqp: no 'actions' in this node.
2024-01-09 16:20:50.545 +0100 Password complexity node not found.
2024-01-09 16:20:52.471 +0100 Error: pan_cfg_get_sysd_bool(pan_cfg_utils.c:7190): failed to fetch cfg.logfwd.need-utf8: NO_MATCHES
2024-01-09 16:20:54.108 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for GPclient
2024-01-09 16:20:54.109 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for Content
2024-01-09 16:20:54.110 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for Antivirus
2024-01-09 16:20:54.110 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for WildFire
2024-01-09 16:20:54.110 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for Content
2024-01-09 16:20:54.223 +0100 Error: pan_query_collection_stats_update_state(pan_log_query_collection_stats.c:329): Couldn't find collection stat for job id 560
2024-01-09 16:20:54.284 +0100 Error: pan_query_collection_stats_update_state(pan_log_query_collection_stats.c:329): Couldn't find collection stat for job id 561
2024-01-09 16:20:54.361 +0100 Error: pan_query_collection_stats_update_state(pan_log_query_collection_stats.c:329): Couldn't find collection stat for job id 562
2024-01-09 16:20:54.368 +0100
##### Non-BATCH report found (custom-dynamic-report)
2024-01-09 16:20:54.391 +0100 client dagger reported op command was SUCCESSFUL
2024-01-09 16:20:54.401 +0100 report generation started for 'custom-dynamic-report'
2024-01-09 16:20:54.401 +0100 ** generating report for time from 1704810054 to 1704813653
2024-01-09 16:20:55.332 +0100 client dagger reported op command was SUCCESSFUL
2024-01-09 16:20:55.474 +0100 client authd reported op command was SUCCESSFUL
2024-01-09 16:20:56.537 +0100 Error: pan_parse_websrvr_tls_service_profile(pan_system_settings.c:3924): missing name parameter
2024-01-09 16:20:58.678 +0100 client device reported Phase 0 was SUCCESSFUL
2024-01-09 16:20:59.663 +0100 client routed reported Phase 1 was SUCCESSFUL
2024-01-09 16:20:59.708 +0100 client ha_agent reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:01.407 +0100 client ikemgr reported Phase 1 FAILED
2024-01-09 16:21:02.920 +0100 client dhcpd reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:03.276 +0100 client varrcvr reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:03.445 +0100 client rasmgr reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:04.275 +0100 client logrcvr reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:04.328 +0100 client sslmgr reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:04.478 +0100 client satd reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:04.522 +0100 client sslvpn reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:04.562 +0100 client authd reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:04.592 +0100 client pppoed reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:04.847 +0100 client dnsproxyd reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:04.883 +0100 client cryptod reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:05.139 +0100 client l2ctrld reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:05.204 +0100 client iotd reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:05.338 +0100 client distributord reported Phase 1 was SUCCESSFUL
2024-01-09 16:21:08.689 +0100 Error: pan_mgmt_client_table_do_commit(pan_cfg_commit_jobs.c:4085): phase 1 failed
2024-01-09 16:21:08.691 +0100 client routed reported error: config commit phase 1 aborted(Module: routed)
2024-01-09 16:21:08.691 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.693 +0100 client device reported error: Config commit phase 1 aborted(Module: device)
2024-01-09 16:21:08.693 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.694 +0100 client ikemgr reported error: panike_daemon phase 1 aborted(Module: ikemgr)
2024-01-09 16:21:08.694 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.697 +0100 client dhcpd reported error: config commit phase 1 aborted(Module: dhcpd)
2024-01-09 16:21:08.697 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.698 +0100 client varrcvr reported error: config commit phase 1 aborted(Module: varrcvr)
2024-01-09 16:21:08.698 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.699 +0100 client sslvpn reported error: modsslvpn phase 1 aborted(Module: sslvpn)
2024-01-09 16:21:08.699 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.700 +0100 client rasmgr reported error: rasmgr phase 1 aborted(Module: rasmgr)
2024-01-09 16:21:08.700 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.701 +0100 client satd reported error: satd phase 1 aborted(Module: satd)
2024-01-09 16:21:08.701 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.705 +0100 client pppoed reported error: config commit phase 1 aborted(Module: pppoed)
2024-01-09 16:21:08.705 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.705 +0100 client dnsproxyd reported error: config commit phase 1 aborted.(Module: dnsproxyd)
2024-01-09 16:21:08.706 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.708 +0100 client cryptod reported error: config commit phase 1 aborted(Module: cryptod)
2024-01-09 16:21:08.708 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.708 +0100 client l2ctrld reported error: config commit phase 1 aborted(Module: l2ctrld)
2024-01-09 16:21:08.708 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.710 +0100 client iotd reported error: config commit phase 1 aborted(Module: iotd)
2024-01-09 16:21:08.710 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:08.714 +0100 Error: pan_cfg_commit_to_local_device(pan_cfg_commit_handler.c:4087): Commit failed
2024-01-09 16:21:08.714 +0100 HTTPD: Abort Commit.
2024-01-09 16:21:08.714 +0100 Error: httpd_abort(httpd_commit.c:69): Failed to remove '/opt/pancfg/mgmt/webui/config_cand.xml'
2024-01-09 16:21:08.715 +0100 client distributord reported error: Config commit phase1 aborted(Module: distributord)
2024-01-09 16:21:08.715 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:09.046 +0100 client useridd reported Phase 1 FAILED
2024-01-09 16:21:09.046 +0100 Error: pan_mgmt_client_p1done_callback(pan_cfg_commit_jobs.c:265): but there was no outstanding Phase 1. Ignoring
2024-01-09 16:21:09.048 +0100 client useridd reported error: Config commit phase 1 aborted(Module: useridd)
2024-01-09 16:21:09.048 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:09.404 +0100 All post P2 commit tasks for job 106 finished with 23 seconds
'cfg.url-vendor-old': NO_MATCHES
/usr/local/bin/bin_scripts/old_url_vendor_is_pan.sh: line 2: [: !=: unary operator expected
/usr/local/bin/bin_scripts/pan_category_exists.sh: line 2: [: too many arguments
2024-01-09 16:21:10.037 +0100 Could not find url vendor, returning paloaltonetworks as default
2024-01-09 16:21:10.058 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for GPclient
2024-01-09 16:21:10.061 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for Content
2024-01-09 16:21:10.065 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for Antivirus
2024-01-09 16:21:10.065 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for WildFire
2024-01-09 16:21:10.067 +0100 Error: pan_get_content_release_date_by_sysd(pan_ops_content.c:1443): Failed to fetch from sysd for Content
2024-01-09 16:21:10.499 +0100 Error: pan_cfg_mgr_get_device_cert_details(pan_cfg_mgr.c:10935): Error getting dev cert validity status from sysd node
2024-01-09 16:21:10.798 +0100 client device reported error: Error: Internal Error(Module: device)
2024-01-09 16:21:10.798 +0100 Error: _pan_mgmt_client_errors_callback(pan_cfg_commit_jobs.c:847): but there was no outstanding Phase 0/Phase 1/Phase 2. Ignoring - verify: 0
2024-01-09 16:21:11.044 +0100 client device reported Phase 1 FAILED
2024-01-09 16:21:11.044 +0100 Error: pan_mgmt_client_p1done_callback(pan_cfg_commit_jobs.c:265): but there was no outstanding Phase 1. Ignoring
Connection to Update server: updates.paloaltonetworks.com completed successfully, initiated by

[RomainDuez]

Little update on this subject.... it was a bug on eve lab. it works with real VMs.
Thanks Claw4609.

[kiwi]

Thanks for the heads up @RomainDuez 👍