VPN client certificates rejected until firewall reboot

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
Please sign in to see details of an important advisory in our Customer Advisories area.
Palo Alto Networks Approved
Palo Alto Networks Approved
Community Expert Verified
Community Expert Verified

VPN client certificates rejected until firewall reboot

L4 Transporter

I had to reboot my firewall this morning because it erroneously rejected client certificates required by a VPN.

Firewall system logs show critical event "Out of memory condition detected, kill process 3" at 4:06am

 

I had the exact same issue on May 5th as well (and reporting to PA) where Clients getting VPN certificate errors despite being nowhere near expiration and reinstalling certifications

 

Is anyone aware of a fix?

1 accepted solution

Accepted Solutions

L7 Applicator

I also have seen this issue. Clients were not able to connect and they were presented with a message that a valid certificste is required. I also saw the out of memory logs. After that I installed PAN-OS 9.1.10 which has quite a few fixes for something that could result in this problem. So far the error did not happen again.

View solution in original post

12 REPLIES 12

Cyber Elite
Cyber Elite

@fhewiufhwefhwe,

I've ran into this a few times with 10.0 throughout various releases and haven't gotten an actual direct answer from support. I'd keep reporting it, because it's definitely a bug somewhere that they just don't appear to have enough data to track down yet.  

L7 Applicator

I also have seen this issue. Clients were not able to connect and they were presented with a message that a valid certificste is required. I also saw the out of memory logs. After that I installed PAN-OS 9.1.10 which has quite a few fixes for something that could result in this problem. So far the error did not happen again.

Are either of you running in HA Pair?  I am wondering whether or not that might mitigate the issue in active-passive and/or active-active until there is a bug fix.  Both times this issue occurred early morning, and fortunately only two people were in the office by then.

I had the issue in a HA pair (active-passive). Actually we have more than 10 other firewall HA pairs where we use global protect, but so far (luckily) the issue only happened on one of them ...

Got it.  So the passive firewall took over while you rebooted the problematic active firewall, and users didn't have downtime during the reboot.  Is that correct?  How much time did it take to configure active-passive mode for the first time?

As long as you immediately reboot the firewall after the OOM systemlog, then yes you will be able to reduce the downtime to almost 0. Otherwise there will still be a timeframe where users are not able to connect.

Setting up a HA pair on the firewallside is quite easy to do. The walkthrough with a step by step manual you can find here: https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-admin/high-availability/set-up-activepassive-ha/...

Depending on thw network setup you need to change some things there too.

 

What PAN-OS version do you currently run on this firewall?

9.1.9

I tried upgrading to 10.0 a couple of times last year, but found it too buggy at the time.  Not sure if it stable enough to run production now, but I will likely wait at least a few more weeks before considering an upgrade.

I think you should consider an update to 9.1.10. Maybe the situation gets also better for you and maybe the issue is already completely resolved in this version

Agreed.  I'll likely try it this weekend

Hi @fhewiufhwefhwe 

did you do the update to 9.1.10 and if so, did the problem happen again since then?

Updated, but the issue occurred between 30 and 50 days uptime after a memory error.  Within waiting two months or a reocurrence, I have no way to confirm that the issue has been fixed.  The release notes did not mention a similar issue.

That could be an issue with time sync b/w MP and DP. You may need to check with NTP servers, if any. Reboot makes both MP and DP clock in sync and for more info follow below.

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000Clh4CAC 

PAN-160744

https://docs.paloaltonetworks.com/pan-os/9-1/pan-os-release-notes/pan-os-9-1-addressed-issues/pan-os-9-1-9-addressed-issues.html 

  • 1 accepted solution
  • 4608 Views
  • 12 replies
  • 0 Likes
Like what you see?

Show your appreciation!

Click Like if a post is helpful to you or if you just want to show your support.

Click Accept as Solution to acknowledge that the answer to your question has been provided.

The button appears next to the replies on topics you’ve started. The member who gave the solution and all future visitors to this topic will appreciate it!

These simple actions take just seconds of your time, but go a long way in showing appreciation for community members and the LIVEcommunity as a whole!

The LIVEcommunity thanks you for your participation!