Groups info for a user-ip-mapping is outdated

Printer Friendly Page

Groups info for a user-ip-mapping is outdated

Symptom

Traffic is hitting an incorrect security rule when the rule has an LDAP group as a source.

The users initiating the traffic are in the given LDAP group and should match a specific allow rule, but instead they are hitting a default deny rule.

 

If a "show user ip-user-mapping" is performed for a user that has the problem, instead of the information about groups the user belongs to, we see "Groups Info is outdated":

 

> show user ip-user-mapping ip 10.176.130.82

IP address: 10.193.88.82 (vsys1)

User: al\ilija

From: UIA

Idle Timeout: 2693s

Max. TTL: 2693s

Groups Info is outdated

 

On a user that is OK, we should see something similar to this output:

 

> show user ip-user-mapping ip 10.193.88.91

IP address:  10.193.88.91 (vsys1)

User:        al\jovan

From:        UIA

Idle Timeout: 6449s

Max. TTL:    6441s

Groups that the user belongs to (used in policy)

Group(s):    cn=domain users,cn=users,dc=al,dc=com

             cn=it_operations,cn=users,dc=al,dc=com

             cn=vpn_users,cn=users,dc=al,dc=com

             cn=\#hash_group,cn=users,dc=al,dc=com

 

Cause

The issue here is with the group being added to a security rule, before the user-id process had a chance to pull the information from the LDAP.

 

This usually happens when:

  • The domain admin creates the group in the LDAP (as a subgroup of an already monitored group on the firewall) and adds the user as a member of it.
  • Before the LDAP group refresh is done on the firewall, the admin creates a security rule using the same group, which he manually adds in the "user" tab of the policy creation.

 

Resolution

To avoid this, after creating a new group on the LDAP server, wait for the group refresh on the firewall to happen and the group to become visible and selectable in the "user" tab of the policy creation process.

Labels (1)
Comments

Hello,

 

I have a quite similar issue with group membership :

 

We have users that don't match correct policy after a while. Looking at the Userids for these users we can see that the group membership has changed (actually less groups are displayed than before) :

 

root@PA3020_M(active)> show user user-ids match-user domain\user

User Name Vsys Groups
------------------------------------------------------------------
domain\user vsys1 cn=terminal servers users,ou=security groups,ou=thisou,dc=company,dc=com
cn=internet,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com
cn=exchange enterprise servers,cn=users,ou=thisou,dc=company,dc=com
cn=domain admins,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com
cn=terminal server users,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com
cn=proxy bluecoat admins,ou=service accounts,ou=thisou,dc=company,dc=com
cn=exchange services,cn=users,ou=thisou,dc=company,dc=com
cn=matrix_v_dev_read,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com
cn=archiveoneusers,ou=archiveone,ou=service accounts,ou=thisou,dc=company,dc=com
cn=qv_admin,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com
cn=domain users,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com
cn=backup operators,cn=builtin,ou=thisou,dc=company,dc=com

 

 

And after a while check for same user results as this :

 

 

rootPA3020_M(active)> show user user-ids match-user domain\user

User Name Vsys Groups
------------------------------------------------------------------
domain\user vsys1 cn=terminal servers users,ou=security groups,ou=thisou,dc=company,dc=com
cn=internet,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com
cn=terminal server users,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com
cn=archiveoneusers,ou=archiveone,ou=service accounts,ou=thisou,dc=company,dc=com
cn=domain users,ou=security groups,ou=city,ou=europe,ou=thisou,dc=company,dc=com

 

 

 

Forcing a group-mapping refresh by using the cli command

"debug user-id refresh group-mapping"

seems to fixe the issue but it s very unpleasant

 

 

Laurent

Hi Laurent,

 

I've seen some similar behaviour in older PAN-OS versions.  What version are you currently running ?

 

LDAP debug logs and useridd logs should be useful in finding the root cause.  Do you have multiple LDAP servers configured ? I've seen something similar happen when a failover happens to another LDAP server and querying the domain users group.

 

Have you opened a support case for this issue already ? If not, then I recommend to do so.  TAC should be able to identify the issue you are experiencing and provide you with a solution or further debugging instructions.

 

Cheers !

-Kim.

So what's the resolution if you've just logged into the CLI and found this "Groups info is outdated" message?

 

"If you find Y, it's because of X, don't do X in future" isn't a resolution, it's a prevention. 

I had similar issue with PA 3050 but in my case LDAP AD admin changed the subgroup of the group mentioned in security policy. He basically did OU path changes  in subgroup couple of days back. When I readded the group in USER ID mappings and commited my changes , the user traffic started hitting correct UID policies. This is frustrating as people obviously  blame fiewalls. Does anyone know that if OU path changes happens then what time it takes to refresh it in PA UID group mapping settings ? 

everything  was woking when OU path changes happened , so I assume PA was using users


@ialeksov wrote:
Symptom Traffic is hitting an incorrect security rule when the rule has an LDAP group as a source.

The users initiating the traffic are in the given LDAP group and should match a specific allow rule, but instead they are hitting a default deny rule.

 

If a "show user ip-user-mapping" is performed for a user that has the problem, instead of the information about groups the user belongs to, we see "Groups Info is outdated":

 

> show user ip-user-mapping ip 10.176.130.82

IP address: 10.193.88.82 (vsys1)

User: al\ilija

From: UIA

Idle Timeout: 2693s

Max. TTL: 2693s

Groups Info is outdated

 

On a user that is OK, we should see something similar to this output:

 

> show user ip-user-mapping ip 10.193.88.91

IP address:  10.193.88.91 (vsys1)

User:        al\jovan

From:        UIA

Idle Timeout: 6449s

Max. TTL:    6441s

Groups that the user belongs to (used in policy)

Group(s):    cn=domain users,cn=users,dc=al,dc=com

             cn=it_operations,cn=users,dc=al,dc=com

             cn=vpn_users,cn=users,dc=al,dc=com

             cn=\#hash_group,cn=users,dc=al,dc=com

 

Cause

The issue here is with the group being added to a security rule, before the user-id process had a chance to pull the information from the LDAP.

 

This usually happens when:

  • The domain admin creates the group in the LDAP (as a subgroup of an already monitored group on the firewall) and adds the user as a member of it.
  • Before the LDAP group refresh is done on the firewall, the admin creates a security rule using the same group, which he manually adds in the "user" tab of the policy creation.

 

Resolution

To avoid this, after creating a new group on the LDAP server, wait for the group refresh on the firewall to happen and the group to become visible and selectable in the "user" tab of the policy creation process.



@ialeksov wrote:
Symptom Traffic is hitting an incorrect security rule when the rule has an LDAP group as a source.

The users initiating the traffic are in the given LDAP group and should match a specific allow rule, but instead they are hitting a default deny rule.

 

If a "show user ip-user-mapping" is performed for a user that has the problem, instead of the information about groups the user belongs to, we see "Groups Info is outdated":

 

> show user ip-user-mapping ip 10.176.130.82

IP address: 10.193.88.82 (vsys1)

User: al\ilija

From: UIA

Idle Timeout: 2693s

Max. TTL: 2693s

Groups Info is outdated

 

On a user that is OK, we should see something similar to this output:

 

> show user ip-user-mapping ip 10.193.88.91

IP address:  10.193.88.91 (vsys1)

User:        al\jovan

From:        UIA

Idle Timeout: 6449s

Max. TTL:    6441s

Groups that the user belongs to (used in policy)

Group(s):    cn=domain users,cn=users,dc=al,dc=com

             cn=it_operations,cn=users,dc=al,dc=com

             cn=vpn_users,cn=users,dc=al,dc=com

             cn=\#hash_group,cn=users,dc=al,dc=com

 

Cause

The issue here is with the group being added to a security rule, before the user-id process had a chance to pull the information from the LDAP.

 

This usually happens when:

  • The domain admin creates the group in the LDAP (as a subgroup of an already monitored group on the firewall) and adds the user as a member of it.
  • Before the LDAP group refresh is done on the firewall, the admin creates a security rule using the same group, which he manually adds in the "user" tab of the policy creation.

 

Resolution

To avoid this, after creating a new group on the LDAP server, wait for the group refresh on the firewall to happen and the group to become visible and selectable in the "user" tab of the policy creation process.




cache to fetch OU path info but everything broke yesterday until i commited already applied configuation. 

I had similar issue with PA 3050 but in my case LDAP AD admin changed the subgroup of the group mentioned in security policy. He basically did OU path changes  in subgroup couple of days back. When I readded the group in USER ID mappings and commited my changes , the user traffic started hitting correct UID policies. This is frustrating as people obviously  blame fiewalls. Does anyone know that if OU path changes happens then what time it takes to refresh it in PA UID group mapping settings ? 

everything  was woking when OU path changes happened , so I assume PA was using users

 

Thanks 

Prakhar

How long does it take to the firewall to refresh the new group in LDAP? Can it be forced?

Gabriel-It depends how huge is your  Based DN scope which is in LDAP profile - DC=xxxcompnay,DC=com...

It can take upto a week as well to refresh group changes, I think you can force it to refesh using CLI command "debug user-id refresh group-mapping", in my case I forced it by removing and then adding group path in User Identification> Group mappings>Group include list>Commit. 

Hi @Gabriel_Linero and @Psrivastava

the default update interval is 60 minutes and is configurable from the group mapping:

update interval.png

 

If it takes a week for you to see new groups, you may be polling a secondary server that is not getting it's updates from it's master in a timely fashion

Thanks a lot @Psrivastava and @reaper!

@reaper

As 60 minutes for group mapping refresh is a rather long period; is there a way to force refresh? In a case when you're changing LDAP tree (moving groups around).

 

Also if I move the same group to a different position in LDAP directory the firewall rule doesn't follow that change; I guess that is intended behaviour? Because a group with the same name in different position in the LDAP tree is a different group.

hi @santonic

yes there is: a 'soft' refresh:

debug user-id refresh group-mapping group-mapping-name <value>
debug user-id refresh group-mapping all 

and a 'hard' refresh

debug user-id reset group-mapping <value>|<all>

and yes indeed, you're targetting a group in a specific location (since we're addressing the group by name, not by unique ID)

Ok, thank you for quick response.