About ben.sellick

Hi all, hoping someone may be able to assist with an issue.   We are seeing that every 3 months our PA device certificate is expiring which causes issues fetching updates from various cloud services (URL filtering, wildfire, update server etc).   Upon renewing the device certificate manually using the OTP in the CSP, the process works and the new certificate is installed fine. It seems just that the automatic renewal process is not working?   I can see the below logs within a tech-support dump that indicate the firewall is aware of the expiring cert, and attempts to renew it +15 days from expiry:   Device_Certgen.log 2022-12-28 04:28:36,218 device_certgen INFO Renewing device certificate 2022-12-28 04:28:37,400 device_certgen INFO Secret_key generated 2022-12-28 04:28:37,400 device_certgen INFO Generated pkey and CSR 2022-12-29 04:33:52,635 device_certgen INFO Renewing device certificate 2022-12-29 04:33:53,493 device_certgen INFO Secret_key generated 2022-12-29 04:33:53,494 device_certgen INFO Generated pkey and CSR 2022-12-30 04:41:24,267 device_certgen INFO Renewing device certificate 2022-12-30 04:41:26,385 device_certgen INFO Secret_key generated 2022-12-30 04:41:26,385 device_certgen INFO Generated pkey and CSR 2022-12-31 04:09:24,314 device_certgen INFO Renewing device certificate 2022-12-31 04:09:26,013 device_certgen INFO Secret_key generated 2022-12-31 04:09:26,013 device_certgen INFO Generated pkey and CSR 2023-01-01 04:42:07,632 device_certgen INFO Renewing device certificate 2023-01-01 04:42:10,039 device_certgen INFO Secret_key generated 2023-01-01 04:42:10,039 device_certgen INFO Generated pkey and CSR 2023-01-02 04:46:43,610 device_certgen INFO Renewing device certificate 2023-01-02 04:46:45,492 device_certgen INFO Secret_key generated 2023-01-02 04:46:45,492 device_certgen INFO Generated pkey and CSR 2023-01-03 04:22:00,381 device_certgen INFO Renewing device certificate 2023-01-03 04:22:01,157 device_certgen INFO Secret_key generated 2023-01-03 04:22:01,157 device_certgen INFO Generated pkey and CSR 2023-01-04 04:40:39,431 device_certgen INFO Renewing device certificate 2023-01-04 04:40:40,627 device_certgen INFO Secret_key generated 2023-01-04 04:40:40,627 device_certgen INFO Generated pkey and CSR 2023-01-05 04:55:41,253 device_certgen INFO Renewing device certificate 2023-01-05 04:55:43,810 device_certgen INFO Secret_key generated 2023-01-05 04:55:43,810 device_certgen INFO Generated pkey and CSR 2023-01-06 04:28:23,482 device_certgen INFO Renewing device certificate 2023-01-06 04:28:25,111 device_certgen INFO Secret_key generated 2023-01-06 04:28:25,111 device_certgen INFO Generated pkey and CSR 2023-01-07 04:30:09,274 device_certgen INFO Renewing device certificate 2023-01-07 04:30:11,694 device_certgen INFO Secret_key generated 2023-01-07 04:30:11,694 device_certgen INFO Generated pkey and CSR 2023-01-08 04:21:50,503 device_certgen INFO Renewing device certificate 2023-01-08 04:21:52,227 device_certgen INFO Secret_key generated 2023-01-08 04:21:52,227 device_certgen INFO Generated pkey and CSR 2023-01-09 04:17:50,467 device_certgen INFO Renewing device certificate 2023-01-09 04:17:52,349 device_certgen INFO Secret_key generated 2023-01-09 04:17:52,349 device_certgen INFO Generated pkey and CSR 2023-01-10 04:27:53,587 device_certgen INFO Renewing device certificate 2023-01-10 04:27:55,029 device_certgen INFO Secret_key generated 2023-01-10 04:27:55,029 device_certgen INFO Generated pkey and CSR 2023-01-11 04:50:00,334 device_certgen INFO Renewing device certificate 2023-01-11 04:50:01,125 device_certgen INFO Secret_key generated 2023-01-11 04:50:01,125 device_certgen INFO Generated pkey and CSR 2023-01-11 16:20:34,528 device_certgen ERROR Device certificate has expired 2023-01-11 16:20:34,565 device_certgen INFO Removing device certificate 2023-01-11 16:20:34,565 device_certgen INFO Removing device certificate 2023-01-11 16:20:34,577 device_certgen INFO Deleting certificates in /opt/pancfg/mgmt/ssl/private (PID: 6020)! 2023-01-11 16:20:34,577 device_certgen INFO Deleting certificates in /opt/pancfg/mgmt/ssl/private (PID: 6020)! 2023-01-11 16:21:37,212 device_certgen INFO Device certificate not found   authd.log 2023-01-11 04:50:03.055 +1100 debug: _device_cert_cb(pan_authd_cas.c:564): change: notify obj 'cfg.device-cert-status', e.g. thermite cert is installed/renewed: was-timestamp xxxxxx ; is-timestamp xxxxxxx 2023-01-11 04:50:03.057 +1100 Device cert (thermite) is renewed, update it in CAS context 2023-01-11 04:50:03.057 +1100 debug: pan_cryptod_sysd_access_key(pan_cryptod_sysd_api.c:1028): Sending GET: id:device_cert_public_key, flag:4 to cryptod 2023-01-11 04:50:03.062 +1100 debug: pan_cryptod_sysd_access_key(pan_cryptod_sysd_api.c:1037): Send GET msg to cryptod for id:device_cert_public_key successful 2023-01-11 04:50:03.062 +1100 debug: pan_cryptod_sysd_access_key(pan_cryptod_sysd_api.c:1057): Received (for id:device_cert_public_key), key data (len=7930): 2023-01-11 04:50:03.062 +1100 debug: pan_cryptod_dump_buf(pan_cryptod_sysd_api.c:749): [xxx] ... 2023-01-11 04:50:03.062 +1100 debug: pan_cryptod_sysd_access_key(pan_cryptod_sysd_api.c:1028): Sending GET: id:device_cert_private_key, flag:4 to cryptod 2023-01-11 04:50:03.064 +1100 debug: pan_cryptod_sysd_access_key(pan_cryptod_sysd_api.c:1037): Send GET msg to cryptod for id:device_cert_private_key successful 2023-01-11 04:50:03.064 +1100 debug: pan_cryptod_sysd_access_key(pan_cryptod_sysd_api.c:1057): Received (for id:device_cert_private_key), key data (len=3271): 2023-01-11 04:50:03.064 +1100 debug: pan_cryptod_dump_buf(pan_cryptod_sysd_api.c:749): [xxx] ... 2023-01-11 04:50:03.085 +1100 debug: _populate_device_cert(pan_authd_cas.c:520): device cert expiry epoch = xxxxxxxxxxx 2023-01-11 04:50:03.085 +1100 debug: _populate_device_cert(pan_authd_cas.c:527): device cert subject = /CN=xxxxxxxxxx/O=Palo Alto Networks/L=Santa Clara/ST=CA/C=US/xxxxxxxxx=tpm/xxxxxx=panos/xxxxxxx=xxxxxxxxxx   The system log does not show any events at the time of attempted renewal.   Communication between the device and PAN-DB cloud services is working normally while the device certificate exists, so I do not believe there is a connectivity/communication problem - however is there a specific log file or URL I can test against?   Any advice on troubleshooting further is greatly appreciated. Thanks. ... View more