GlobalProtect - Authentication Issues

[AdamSC]

Hi all,

Fairly new to PAN and in the process of an ASA migration. Despite TAC/VAR assistance, I'm still having some issues with my GlobalProtect user experience. Fortunately it's not in production yet but the feedback has been inconsistent.

Business Requirements:

-Use GlobalProtect to tunnel all external user traffic back to HA pair for web filtering/visibility

-Only enterprise devices can connect; use existing PKI to validate

-Approved users can manually switch to an "Internal VPN" gateway to access restricted resources

-These approved users will go thru a OTP leveraging soft tokens

-Users in the office should not have to enter credentials to connect, but their GP client should connect for accurate User-ID information

Current Portal Config:

-1 portal configured with an authentication profile linking to Cisco ISE; strictly AD check, no OTP

-The portal is configured for a certificate profile (internal CA but no usernames)

-The portal generates/accepted a 24 hour cookie for authentication override

-Manual gateways are configured for dynamic OTP (instead of passing the credentials)

-There are 2 agent configurations:

        1. First config allows access to a single gateway that only allows web traffic for visibility

        2. Second config allows the web traffic gateway and an internal resources gateway for true "VPN" experience

-The app is configured for SSO and User-Logon (Always On) mode; cannot be disabled by user and must connect for network access (these settings are to meet the business requirement of preventing unfettered Web access for remote associates)

Current Gateway Config:

-The Web Filter gateway is pointing to Cisco ISE (I expect this to recycle the portal credentials without issue)

-The Internal VPN gateway points to Azure MFA:

        1. This is our only means of leveraging MFA; the same AD check should succeed and issue a challenge if the user

             is enrolled in MFA

        2. I do not expect to recycle credentials as this is accessed via manual selection

                A. Portal configuration specifies that this component requires OTP/MFA

-I do have an Internal Gateway via 'Internal Host Detection' and it also points at Cisco ISE 

        1. This is working fine; the macOS clients do not get SSO, as the GP app config option is for Windows only

Issues:

-Sometimes we receive multiple password prompts and OTP prompts

-I do not expect to receive a password prompt due to the SSO option, but sometimes do when connecting 

-The OTP prompt is 'approve' so I am certain duplicate prompts are not due to typos

-I occassionally do not see ISE entries after successful authentication (which I should see when connecting to the portal) but this may be due to authentication cookies being generated/accepted

Questions:

-The default behavior would pass the AD credentials used in the Portal to the Gateway, correct?

-Are credentials passed from one gateway to another?

        1. As mentioned, I have an initial gateway that everyone ends up on; some users can manually select another gateway

                A. This process always results in a password prompt followed by a OTP prompt

-Exactly what does the Authentication Override option do?

        1. I have read documentation that it is for fewer PW/OTP prompts, but how do I verify it is working?

                A. I have never been able to logout/login without a password prompt/OTP, for reference

-How do reboots/returning from sleep/hibernation effect the user experience?

I can post screenshots and elaborate further if required.

I have already gone through the following links:

-https://www.paloaltonetworks.com/documentation/80/globalprotect/globalprotect-admin-guide/authentica...

-https://www.paloaltonetworks.com/documentation/80/globalprotect/globalprotect-admin-guide/authentica...

We are running PAN-OS 8.1.4 and GP 4.1.6.12 on PA5250s.

Thank you

[Remo]

Hi @AdamSC

Congratulations, directly with your first post here in the great Live community you have reached the champions league :Pfor global protect configurations (SSO, MFA, Always-On, Enforce GP for Network Access). Unfortunately I am dealing with such a configuration since one year and so far I still need to have workarounds configured to have it working without GP being a too big pain for the users.

Let's start. So far SSO and MFA is actually a bad idea except if you ONLY use TOTP (or something else that does not result in a direct notification for the users). The issue here is, global protect sends an authentication to the portal as soon the login is done. Unfortunately this is done in the background and at this stage global protect is not ready to show an OTP prompt to the user so the authentication fails directly. Immediately the next authentication attempt (with the credentials GP has because of SSO) starts but this time the additional prompt is shown to the user and as you describe it the user has already received multiple MFA/OTP notifications. When we first experianced this issue one year ago there were some meetings with paloalto and the result was a feature request where we hope will be implemented in the next GP major version. With this feature request the behaviour you need should be implemented without bothering the users with too much useless notifications. Unfortunately this explanation does not really help you, so I try to explain how I solved this...

On the portal I use autentication override with a cookie that is issues and used only on the portal so here the user has to log in only once and starting after this login the cookie is used to authenticate the user to the portal. On the gateway I activated the setting that requires dynamic passwords for external gateways. The tradeoff is that for every login from external users have to enter username, password and OTP. But so far this is the best working config I have found when the requirements are: SSO, MFA, Always-On and enforce GP. (If someone has a better solution PLEASE share it 😛   )

[Remo]

---

@AdamSC wrote:

The default behavior would pass the AD credentials used in the Portal to the Gateway, correct?

---

Yes, but in case of SSO these credentials are passed to portal and gateway.

---

@AdamSC wrote:
-Are credentials passed from one gateway to another?

---

Not from one gateway to another but from SSO to everything else.

---

@AdamSC wrote:

1. As mentioned, I have an initial gateway that everyone ends up on; some users can manually select another gateway

                A. This process always results in a password prompt followed by a OTP prompt

---

Do you have the setting "require dynamic password" enabled for external gateways?

---

@AdamSC wrote:

-Exactly what does the Authentication Override option do?

---

It gives you the possibility to issue cookies to users when they successfully login. With the application override you can also specify how long these cookies are valid so they can be used for aurhentication instead of requiring rhe user to enter username/password every time. An example is if you want to have a vpn access from external with MFA. Without application override the user hase to log in to the gateway and to the portal. With the application override a cookie is issued after MFA login on the portal. This cookie is then used on the gateway which result in no additional MFA login on the gateway.

---

@AdamSC wrote:

1. I have read documentation that it is for fewer PW/OTP prompts, but how do I verify it is working?

---

In the system logs on the firewall you can see if a cookie was used for authentication. Or test it on a gateway that required MFA. Issue a cookie that is valid for a specified time and also accept it for authentication. In the specified timeframe you will not be asked for an OTP (if require dynamic passwords is disabled.

---

@AdamSC wrote:

A. I have never been able to logout/login without a password prompt/OTP, for reference

---

Did you enable also the setting to accept cookies on either the portal or gateway?

---

@AdamSC wrote:

-How do reboots/returning from sleep/hibernation effect the user experience?

---

Good question where I don't have consistent answer. It depends on the hardware/driver, software installed - specially with additional credential provider and/or if there was a network change after wake up. Probably even more, bur most of the time it works properly and sometime there are these strange situations that noone knows a solution 

[Brandon_Wertz]

I'm actually in a fairly similar situation...Migrating from Cisco 5580s to 5220 with GP.  We're running 8.0.12 with 4.1.6 GP

We have the following business requirements:

- Pre-Login

- Always On

- Machine cert Based auth

- Internal host detection so tunnel isn't established when inside forcing all traffic through it

So it almost feels like we have the same requirements, but you do have a few extras in there.

I will say that TAC wasn't helpful in understanding what I was trying to get working, or maybe I just didn't know how to relay my needs.  I just this working through trial and error and 4 different support cases.  In the end for me it just came down to using the correct cert profiles for the correct use case.

One potential issue I can see is if you're planning on using machine cert for auth then you're going to have issues defining different policies for users, so just keep that in mind. 

[AdamSC]

I can see cookie authentication in the logs, so that must be working.

I think one thing that's different here is that I am not doing MFA on the portal, but am on one single gateway. The business essentially wants people to be able to turn their laptops on and connect transparently (assuming the machine certificate check is valid and the SSO credentials succeed) for web access only. A lot of employees do not need internal resources, but if they did, the second gateway (manually selected) prompts for a password + MFA (Microsoft Authenticator). 

I've attached a few screenshots of my configuration. From what I can tell:

-The Web Filter VPN has the highest priority and is connected to automatically. It does not ask for credentials

        Q: Is this because of the authentication cookie, or is it due to the default behavior of passing credentials?

-The Internal VPN is manual only and the configuration states that it does require dynamic passwords. When selecting this, I enter my password and then receive an MFA prompt on my phone. Do note that this RADIUS server is not the same one as the portal.

        Q: Is this expected behavior?

It looks like my gateways are not configured to accept a cookie, so technically speaking, what is driving the auto-login from portal to "Web Filter" gateway? I was believing the SSO option under Portal > Agent > Agent Config > App was essentially providing SSO to the portal from Windows (Kerberos). If that is disabled, I am still not expecting to enter my password in for the Web Filter gateway (but will for the portal). I'll test that shortly.

Also, my certificate profile is strictly looking for the presence of a machine cert signed by the internal CA. It's not using it for 'real' authentication and does not populate the username field. 

I'm curious about the logs. It seems like there's a lot of redundancy with authentication/logins. Is this normal?

I greatly appreciate the help.

 Portal1Portal2Portal3GW1GW2GW3Cert_Profile

Log1Log2Log3Log4Log5

[asilliker]

You are currently configured to only do cookie authentication on the portal. You'll see on the web filter gateway that your authentication has been done via an authentication profile. 

The transparent authentication to the web filter gateway is done by replaying the user/pass received in the client by either your manual entry of the values or the "SSO" settings. "SSO" in the case is credential provider wrapping, not kerberos SSO. Unless you have generated an assigned a kerberos keytab to your authentication profile(s) in use you are not using kerberos SSO. 

The dynamic passwords for manual only setting is why you are being prompted to provide your password when you attempt a manual connection. If your RADIUS server is using access-challenge to trigger the sending of the push message, then  you may not require this and would otherwise just receive a push so long as the user/password for the internal VPN gateway is the same as for the portal. 

Your logging output volume is normal. 

Your current portal configuration should only require you to authenticate via RADIUS every 24 hours. Whether or not you have "SSO" enabled or Save User Credentials (currently no) set will govern whether you are prompted if you disable cookie authentication. 

[AdamSC]

Thank you for this clarification, it helps greatly.

Can you clarify the "credential provider wrapping" part? And how it relates to only being compatible with Windows? I have not defined any Kerberos SSO. I'm a little confused as to what 'magic' enables SSO to function.

The username/password is the same for the Azure MFA server (as for the ISE server hit for the portal connection), and honestly I would rather just respond to the Access-Challenge instead of providing duplicate credentials.

This thread has helped me understand the various options and the flow, but I do have two pain points still that I'm unable to figure out.

1. I sometimes see my connection "loop" and by this I mean that I will select the 'Internal VPN' gateway and provide credentials + approve the MFA prompt and connect. But then, I'm dropped back into the 'Web Filter' gateway for seemingly no reason. It does not always happen but my logs confirm the repeated logout/logins.

2. I sometimes see authentication failures on the 'Internal VPN' gateway but the Azure logs show successes. When these messages appear, the user experience is several MFA approvals/prompts until it eventually works. The Azure MFA seemingly works via an on-prem NPS box that calls to Azure AD cloud. The timestamps between those two components are very tight, so it doesn't seem to be a latency issue. The NPS box is on the same network segment as the PAN, so that should not be an issue either. My RADIUS server profile settings are the recommended 60 seconds/3 re-tries.

Once again I greatly appreciate the help.

[AdamSC]

Thanks for the reply!

Very similar requirements, but I'm sort of faking the machine certicate authentication. It's mostly used as a check to verify company hardware, but the CN/SAN is not used for authentication (like we would do internally via 802.1x). I vouched this works by trying on a personal computer. The only downside is the initial connection does not know which certificate store to look into and results in a prompt showing user certificates (again, works for me but not ideal). This is corrected wants proper GP app settings are downloaded.

[AdamSC]

So I attempted to remove the checkbox from the manually-selected gateways requiring dynamic password, figuring it might work since the same username/password is used before issuing the MFA prompt. I did not have desired results.

I rebooted, hit the 'Internal VPN' gateway, but was still met with a prompt to enter my username/password. I checked the logs and see an authentication failure. I entered them in, then had to approve the MFA prompt on my phone. I did, but received a duplicate prompt.

I did a capture on the NPS box, but never see a RADIUS access-challenge. Not sure how this works. I do see it reach out to public Microsoft IP address between the RADIUS-request and the RADIUS-accept messages, though. I assume this relates to the logs I see when inside the Azure portal. I do not see any RADIUS messages when doing a capture on the PAN; it's all TLS traffic.

PCAP on NPS

Is there some reason this wouldn't work? 

I also observed today that when my colleague needed to enter his MFA 3 times, there was some delay between the successful login message in the PAN logs and the successful authentication in the Azure portal logs:

20 second delay

52 second delay

11 second delay

Not sure what the reason is for this but perhaps it's an issue.

[AdamSC]

I believe we have this issue narrowed down a bit.

Previously, I was seeing periodic authentication failures due to timeout. TAC had me change the timeout value via CLI to 50 seconds (from the default 25 seconds) because periodically the round-trip for the authentication exceeded 25 seconds. This wasn't common but it happened on a few occassionals.

I know see successes and the round-trip is generally 3-6 seconds between the RADIUS Access-Request and the RADIUS Access-Accept packets sent/received on the PAN firewall. The issue is, it often 'loops'.

I'll see a user login to the portal via ISE and then the 'Web Filter' gateway via cookie without issue (normally). Then they select the 'Internal VPN' and there are success messages in the PAN and the Azure logs. However, briefly after (usually 11 seconds as per the logs) they are disconnected from that gateway and dropped back into the 'Web Filter' gateway.

Has anyone seen this behavior before? TAC keeps asking for more and more debugs/captures/etc. but it doesn't feel like we're going to resolve it.