VPN client certificates rejected until firewall reboot

[fhewiufhwefhwe]

I had to reboot my firewall this morning because it erroneously rejected client certificates required by a VPN.

Firewall system logs show critical event "Out of memory condition detected, kill process 3" at 4:06am

I had the exact same issue on May 5th as well (and reporting to PA) where Clients getting VPN certificate errors despite being nowhere near expiration and reinstalling certifications

Is anyone aware of a fix?

[BPry]

@fhewiufhwefhwe,

I've ran into this a few times with 10.0 throughout various releases and haven't gotten an actual direct answer from support. I'd keep reporting it, because it's definitely a bug somewhere that they just don't appear to have enough data to track down yet.  

[vsys_remo · Accepted solution]

I also have seen this issue. Clients were not able to connect and they were presented with a message that a valid certificste is required. I also saw the out of memory logs. After that I installed PAN-OS 9.1.10 which has quite a few fixes for something that could result in this problem. So far the error did not happen again.

[fhewiufhwefhwe]

Are either of you running in HA Pair?  I am wondering whether or not that might mitigate the issue in active-passive and/or active-active until there is a bug fix.  Both times this issue occurred early morning, and fortunately only two people were in the office by then.

[vsys_remo]

I had the issue in a HA pair (active-passive). Actually we have more than 10 other firewall HA pairs where we use global protect, but so far (luckily) the issue only happened on one of them ...

[fhewiufhwefhwe]

Got it.  So the passive firewall took over while you rebooted the problematic active firewall, and users didn't have downtime during the reboot.  Is that correct?  How much time did it take to configure active-passive mode for the first time?

[vsys_remo]

As long as you immediately reboot the firewall after the OOM systemlog, then yes you will be able to reduce the downtime to almost 0. Otherwise there will still be a timeframe where users are not able to connect.

Setting up a HA pair on the firewallside is quite easy to do. The walkthrough with a step by step manual you can find here: https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/high-availability/set-up-activepassive-ha/...

Depending on thw network setup you need to change some things there too.

What PAN-OS version do you currently run on this firewall?

[fhewiufhwefhwe]

9.1.9

I tried upgrading to 10.0 a couple of times last year, but found it too buggy at the time.  Not sure if it stable enough to run production now, but I will likely wait at least a few more weeks before considering an upgrade.

[vsys_remo]

I think you should consider an update to 9.1.10. Maybe the situation gets also better for you and maybe the issue is already completely resolved in this version

[fhewiufhwefhwe]

Agreed.  I'll likely try it this weekend