About James_Mucklin

When committing a template only change from panorama to managed firewalls in a HA pair the commit fails.  When committing a template change along with a device group change it succeeds. Template only changes commit fine when being pushed down to managed standalone firewalls. All devices are running PAN-OS 10.1.5-h2 Reviewed the panorama logs along with the logs from the managed firewalls. From the config daemon logs in Panorama there looks to be an issue with the underlying database. When a template only commit is pushed, the logs show Panorama failing to obtain operational logs required in the system daemon. Error messages seen in the logs:   From the configd.log there’s a clear pattern of events;     the commit is pushed from Panorama   2022-05-27 10:26:30.970 +0100 Commit job enqueued. type=2 2022-05-27 10:26:30.973 +0100 start pan_commit_get_cfg_root 2022-05-27 10:26:31.048 +0100 Json array size is 0, nothing will be synced to db   The firewall implies there are issues with the HA database objects when it tries to sync;   2022-05-27 10:26:31.048 +0100 Json array size is 0, nothing will be synced to db 2022-05-27 10:26:31.365 +0100 Error:  pan_cfg_get_oplog_from_sysd_obj(pan_cfg_ha_db_sync.c:539): Unable to find the op value in peer.ha.lib.mgmt.impl.usr.base.mdb-oplog; ignoring 2022-05-27 10:26:31.415 +0100 Return detail-ver 10.1.5 2022-05-27 10:26:32.050 +0100 Json array size is 0, nothing will be synced to db 2022-05-27 10:26:32.368 +0100 Error:  pan_cfg_get_oplog_from_sysd_obj(pan_cfg_ha_db_sync.c:539): Unable to find the op value in peer.ha.lib.mgmt.impl.usr.base.mdb-oplog; ignoring 2022-05-27 10:26:32.604 +0100 start pan_cfg_save_commit_candidate 2022-05-27 10:26:33.054 +0100 Json array size is 0, nothing will be synced to db   This then fails and reports the failure in the log.   2022-05-27 10:17:09.668 +0100 SEATTLETIME: Time to PROCESSJOB:pan_cfg_commit_to_local_device: 22 secs 2022-05-27 10:17:09.673 +0100 Error:  pan_cfg_replaydb_update_status_by_tids(pan_cfg_replaydb.c:624): pan_cfg_replaydb_update_status_by_tids: List of TIDS is empty 2022-05-27 10:17:09.736 +0100 Json array size is 0, nothing will be synced to db 2022-05-27 10:17:09.841 +0100 Warning:  sc3_sendRegInfo(sc3_register.c:411): SC3R: AK not present. 2022-05-27 10:17:10.049 +0100 client dagger reported op command FAILED   The main error that appears over and over is;   2022-05-27 10:19:00.347 +0100 Error:  pan_cfg_get_oplog_from_sysd_obj(pan_cfg_ha_db_sync.c:539): Unable to find the op value in peer.ha.lib.mgmt.impl.usr.base.mdb-oplog; ignoring 2022-05-27 10:19:01.006 +0100 Json array size is 0, nothing will be synced to db --------   Now looking at the firewalls themselves, I can see the ‘client’ side of these errors;   2022-05-27 10:20:17.837 +0100 client dagger reported op command FAILED 2022-05-27 10:20:17.982 +0100 client authd reported op command FAILED 2022-05-27 10:20:18.501 +0100 client dagger reported op command FAILED 2022-05-27 10:20:19.460 +0100 client useridd reported op command FAILED 2022-05-27 10:20:19.672 +0100 client useridd reported op command FAILED 2022-05-27 10:20:19.718 +0100 client dagger reported op command FAILED 2022-05-27 10:20:19.720 +0100 client useridd reported op command FAILED 2022-05-27 10:20:19.930 +0100 client authd reported op command FAILED 2022-05-27 10:20:20.524 +0100 client dagger reported op command FAILED 2022-05-27 10:20:21.341 +0100 client dagger reported op command FAILED 2022-05-27 10:20:21.442 +0100 client authd reported op command FAILED 2022-05-27 10:20:21.921 +0100 client dagger reported op command FAILED 2022-05-27 10:20:22.449 +0100 client useridd reported op command FAILED 2022-05-27 10:20:22.646 +0100 client useridd reported op command FAILED 2022-05-27 10:20:22.691 +0100 client useridd reported op command FAILED   At this point, it looks like Panorama is attempting to push the config down the both managed firewalls in the HA pair, but get stopped because of a database syncing issue. But this still doesn’t explain why the commit all seems to work fine when bundled in with a device group push…..   Is this a bug in 10.1.5 ?  ... View more