topic Re: VPN client certificates rejected until firewall reboot in General Topics

VPN client certificates rejected until firewall reboot

fhewiufhwefhwe — Tue, 29 Jun 2021 20:24:13 GMT

I had to reboot my firewall this morning because it erroneously rejected client certificates required by a VPN.

Firewall system logs show critical event "Out of memory condition detected, kill process 3" at 4:06am

I had the exact same issue on May 5th as well (and reporting to PA) where Clients getting VPN certificate errors despite being nowhere near expiration and reinstalling certifications

Is anyone aware of a fix?

Re: VPN client certificates rejected until firewall reboot

BPry — Tue, 29 Jun 2021 21:11:08 GMT

@fhewiufhwefhwe,

I've ran into this a few times with 10.0 throughout various releases and haven't gotten an actual direct answer from support. I'd keep reporting it, because it's definitely a bug somewhere that they just don't appear to have enough data to track down yet.

Re: VPN client certificates rejected until firewall reboot

Remo — Tue, 29 Jun 2021 22:12:07 GMT

I also have seen this issue. Clients were not able to connect and they were presented with a message that a valid certificste is required. I also saw the out of memory logs. After that I installed PAN-OS 9.1.10 which has quite a few fixes for something that could result in this problem. So far the error did not happen again.

Re: VPN client certificates rejected until firewall reboot

fhewiufhwefhwe — Tue, 29 Jun 2021 22:18:07 GMT

Are either of you running in HA Pair? I am wondering whether or not that might mitigate the issue in active-passive and/or active-active until there is a bug fix. Both times this issue occurred early morning, and fortunately only two people were in the office by then.

Re: VPN client certificates rejected until firewall reboot

Remo — Tue, 29 Jun 2021 22:25:46 GMT

I had the issue in a HA pair (active-passive). Actually we have more than 10 other firewall HA pairs where we use global protect, but so far (luckily) the issue only happened on one of them ...

Re: VPN client certificates rejected until firewall reboot

fhewiufhwefhwe — Tue, 29 Jun 2021 22:29:22 GMT

Got it. So the passive firewall took over while you rebooted the problematic active firewall, and users didn't have downtime during the reboot. Is that correct? How much time did it take to configure active-passive mode for the first time?

Re: VPN client certificates rejected until firewall reboot

Remo — Tue, 29 Jun 2021 23:24:24 GMT

As long as you immediately reboot the firewall after the OOM systemlog, then yes you will be able to reduce the downtime to almost 0. Otherwise there will still be a timeframe where users are not able to connect.

Setting up a HA pair on the firewallside is quite easy to do. The walkthrough with a step by step manual you can find here: https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/high-availability/set-up-activepassive-ha/configure-activepassive-ha.html

Depending on thw network setup you need to change some things there too.

What PAN-OS version do you currently run on this firewall?

Re: VPN client certificates rejected until firewall reboot

fhewiufhwefhwe — Tue, 29 Jun 2021 23:29:31 GMT

9.1.9

I tried upgrading to 10.0 a couple of times last year, but found it too buggy at the time. Not sure if it stable enough to run production now, but I will likely wait at least a few more weeks before considering an upgrade.

Re: VPN client certificates rejected until firewall reboot

Remo — Tue, 29 Jun 2021 23:31:15 GMT

I think you should consider an update to 9.1.10. Maybe the situation gets also better for you and maybe the issue is already completely resolved in this version

Re: VPN client certificates rejected until firewall reboot

fhewiufhwefhwe — Tue, 29 Jun 2021 23:32:26 GMT

Agreed. I'll likely try it this weekend

Re: VPN client certificates rejected until firewall reboot

Remo — Mon, 05 Jul 2021 10:23:17 GMT

Hi @fhewiufhwefhwe

did you do the update to 9.1.10 and if so, did the problem happen again since then?

Re: VPN client certificates rejected until firewall reboot

fhewiufhwefhwe — Tue, 06 Jul 2021 13:51:36 GMT

Updated, but the issue occurred between 30 and 50 days uptime after a memory error. Within waiting two months or a reocurrence, I have no way to confirm that the issue has been fixed. The release notes did not mention a similar issue.

Re: VPN client certificates rejected until firewall reboot

asangra — Tue, 06 Jul 2021 14:13:08 GMT

That could be an issue with time sync b/w MP and DP. You may need to check with NTP servers, if any. Reboot makes both MP and DP clock in sync and for more info follow below.

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000Clh4CAC

PAN-160744

https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-release-notes/pan-os-9-1-addressed-issues/pan-os-9-1-9-addressed-issues.html