Global Protect VPN Client not authenticating to 2012 R2 Domain Controller

[AZCommerceAuthority]

We recently installed a new Domain Controller and can't get the Global Protect VPN client LDAP authentication to work. We have tried everything we can think of and would like to know if anyone else has had this issue and could assist with a solution.

DC01 - Primary DC which firewall and VPN currently authenticates to.

DC02 - Secondary DC which will work as standalone authentication server in testing.

DC03 - New DC which is not allowing authentication from Global Protect VPN client. After entering username and password the client just goes back to the password prompt like nothing happened. 

Any assistance would be greatly appreciated.

Thanks

[dmifsud]

I would check the system log with filter (type eq auth), and also the authd log from the CLI (less mp-log authd.log) as a starting point.

[TomYoung]

Hi @AZCommerceAuthority ,

The CLI "test" command is great to troubleshoot authentication profiles.  https://docs.paloaltonetworks.com/pan-os/10-1/pan-os-admin/authentication/test-authentication-server...  It normally gives the specific reason the authentication fails (bad password, cannot connect to server, etc.).

Thanks,

Tom

[AZCommerceAuthority]

So I had some time to finally test this again and do some log file tracing.

Here is a taste of what I am seeing now when trying to authenticate to the new DC. 

2023-01-12 13:05:39.470 -0700 debug: _retrieve_svr_ids(pan_auth_service.c:648): find auth server id vector for AD-vsys1
2023-01-12 13:05:39.470 -0700 debug: _start_sync_auth(pan_auth_service_handle.c:606): recreate 0th LDAP session to remote server 10.0.2.17:636 after retry-interval (60 sec) has elapsed
2023-01-12 13:05:39.470 -0700 debug: _recreate_a_ldap_session(pan_auth_service_handle.c:418): re-create ldap session (ip ; sourceAddr ; sourceAddr6 ; vsys shared)
2023-01-12 13:05:39.470 -0700 debug: pan_authd_ldap_init_set(pan_authd_shared_ldap.c:917): useLDAPs: 1, startTLS: 0, b_ssl: 1
2023-01-12 13:05:39.470 -0700 ldap uri: ldaps://10.0.2.17:636
2023-01-12 13:05:39.470 -0700 Succeed to init LDAPp=0xff60001330 for entry 0
2023-01-12 13:05:39.470 -0700 b_ssl: Yes
2023-01-12 13:05:39.470 -0700 debug: pan_authd_ldap_bind(pan_authd_shared_ldap.c:634): binding with binddn CN=svc_paloalto,OU=Service Accounts,DC=ourdcname,DC=com
2023-01-12 13:05:39.473 -0700 Error: pan_authd_ldap_bind(pan_authd_shared_ldap.c:643): Failed to bind ldap (Can't contact LDAP server)
2023-01-12 13:05:39.473 -0700 Error: pan_auth_create_a_ldap_session(pan_auth_svr_cctxt.c:2002): Failed to bind, get out
2023-01-12 13:05:39.474 -0700 Error: _recreate_a_ldap_session(pan_auth_service_handle.c:538): failed to re-create 0th LDAP session for server: 10.0.2.17:636
2023-01-12 13:05:39.474 -0700 LDAP auth server 10.0.2.17 is down !!!

On the new DC I have disabled the OS Firewall and tried to connect and it still fails so I know there is not some FW rule allowing port 636 to pass through.

Any ideas on what my issue is? I have tested LDAP on the new server using the service account user and it works perfectly.

[Adrian_Jensen]

Where are you connecting to the DC from? I.e. is the LDAP service running from the management port (default) or have you configured it on the dataplane? (Device->Setup->Services->Service Route Configuration). I am assuming that 10.0.2.17 is NOT an IP on the PA itself that is being NAT'd to another network, correct?

It looks like the PA can't even reach the DC server. If using LDAP from the management port (the default), does the management network have a route to 10.0.2.x? Does the DC have a firewall restricting access to certain source IPs? Can you ping all of the DCs from the management interface  (PA> ping source [mgmt_ip] host 10.0.2.17)?

[AZCommerceAuthority]

Where are you connecting to the DC from? Management Port

I.e. is the LDAP service running from the management port (default) or have you configured it on the dataplane? (Device->Setup->Services->Service Route Configuration).

I am assuming that 10.0.2.17 is NOT an IP on the PA itself that is being NAT'd to another network, correct? YES

It looks like the PA can't even reach the DC server. If using LDAP from the management port (the default), does the management network have a route to 10.0.2.x? YES

Does the DC have a firewall restricting access to certain source IPs? NO - We tested LDAP login from the FW to the DC after turning off the DC FW.

Also I can ping both ways with no issue. Here is ping from server:

Here are some commands I ran for troubleshooting and the results of each command:

admin@ACA-PHX-FW01> show log system direction equal backward eventid equal "auth-server-down"
Time Severity Subtype Object EventID ID Description
===============================================================================
2023/01/12 14:19:06 critical auth auth-se 0 LDAP auth server 10.0.2.17 is down !!!

admin@ACA-PHX-FW01> debug authentication connection-show protocol-type LDAP

conn_id: 65576 ; server addr: 10.0.2.10:636 ; method: LDAP (active directory) ; auth prof name: AD ; own addr: ; vsys: shared ; retry interval: 60 secs ; debug: off ; debug prefix: ;
be up: yes ;
be in retry interval (conn 0): no last connection attempt time: Fri Jan 13 06:07:41 2023
be in retry interval (conn 1): no
be in retry interval (conn 2): no
be in retry interval (conn 3): no
# of received requests: 532
# of sent out requests: 532
# of received responses: 532
# of timed out requests: 0
# of stale responses: 0
last request sent time: Fri Jan 13 09:07:42 2023
last response receive time: Fri Jan 13 09:07:42 2023
conn_id: 65577 ; server addr: 10.0.2.11:636 ; method: LDAP (active directory) ; auth prof name: AD ; own addr: ; vsys: shared ; retry interval: 60 secs ; debug: off ; debug prefix: ;
be up: yes ;
be in retry interval (conn 0): no
be in retry interval (conn 1): no
be in retry interval (conn 2): no
be in retry interval (conn 3): no
# of received requests: 0
# of sent out requests: 0
# of received responses: 0
# of timed out requests: 0
# of stale responses: 0
conn_id: 65578 ; server addr: 10.0.2.17:636 ; method: LDAP (active directory) ; auth prof name: AD ; own addr: ; vsys: shared ; retry interval: 60 secs ; debug: off ; debug prefix: ;
be up: yes ;
be in retry interval (conn 0): no
be in retry interval (conn 1): no
be in retry interval (conn 2): no
be in retry interval (conn 3): no
# of received requests: 0
# of sent out requests: 0
# of received responses: 0
# of timed out requests: 0
# of stale responses: 0

I want to remove the 2 working DC's and only have 10.0.2.17 as LDAP server then run debug authentication connection-show protocol-type LDAP

I definitely feel like this is a permissions issue on the server somewhere. Interestingly enough the server monitoring is only working on DC01. I went through the WMI setup from the documentation and it still is not working on DC02 and DCA:

I did see an article that talks about using WinRM versus WMI which I plan to test later and see if I can get the new server to connect:

https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA14u000000wkkfCAA

[AZCommerceAuthority]

We disabled ssl and used port 389 and can connect to DCA without issue so there this was a certificate issue the whole time. Go ahead and close this thread out and thank you for all your kind and informative replies. We do appreciate everyone trying to help us out.

[AZCommerceAuthority]

We got our FW to work with Global Protect on DCA (new domain controller) without using SSL by just using port 389 however it will not connect on port 636 using SSL now. Any idea what may be missing? 

When we add our new DC and set it as LDAP server using port 636 then check the ssl box, not only will it not authenticate, but in server manager we get the following error: I have checked the certificate store on the working domain controller and not able to find any of the certs that we have on the FW so not sure exactly what I am missing here other than we just inherited this setup and are trying to figure out how to update our domain controllers. 

The SSL server credential's certificate does not have a private key
information property attached to it. This most often occurs when a
certificate is backed up incorrectly and then later restored.
This message can also indicate a certificate enrollment failure.