Tips for finding Knowledge Articles

  • - Enter just a few key words related to your question or problem
  • - Add Key words to refine your search as necessary
  • - Do not use punctuation
  • - Search is not case sensitive
  • - Avoid non-descriptive filler words like "how", "the", "what", etc.
  • - If you do not find what you are looking for the first time,reduce the number of key words you enter and try searching again.
  • - Minimum supported Internet Explorer version is IE9
Home  >
article

KB-3517: adclient consumes high CPU

Centrify DirectControl ,  

20 March,17 at 11:48 PM

Applies to: Centrify DirectControl 5.1.x on all platforms
 
Problem:
A high I/O is reported on the /var file system and the system CPU I/O wait continuouly rises.

Restarting DirectControl and DirectAudit, or running adflush is needed to bring the I/O is back to normal. 
 
Cause:
This is a known issue in pagedSearch logic which causes Centrify's adclient to think that the search result in the local cache is unhealthy, this then causes the process to do a fresh search in AD over and over again.
 
For every new search response from AD, the returned result is always appended (instead of replacing) to the previous results stored in the local cache file, thus causing the local cache file size to keep growing. 
 
This growing cache file size leads to the high CPU symptom.
 
A good way to confirm is to get a listing of /var/centrifydc/gc.cache and /var/centrifydc/dc.cache.
 
As seen in debug log (extract below), there will always be search for keywords=Version:Centrify.Vegas.Installation.1 every 30 seconds.
 
The clues to the problem is:
 
1) Customers will always see "cache store" after it.
 
2) The gc.cache is >2 x the size of dc.cache - and keeps growing
 
3) The CPU and IO will spike every 30 seconds, while in debug log there is no abnormal activity.
 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > lrpc.adobject new object: 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > base.bind.ad Connected root=DC=ad,DC=yourdomain,DC=com, domain=yourdomain.com functionality=2 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > base.bind.ad Address of yourdomain400.yourdomain.com is 10.6.11.142 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > base.bind.ad Performing LDAP binding with GSSAPI mechanisms to server - yourdomain400.yourdomain.com 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > audittrail.internal Invalid value for userName. 
 
Sep 20 12:49:13 fileserver adclient[2135]: DIAG <fd:17 CAPILdapPagedSearch > base.bind.ad Connected to GC in preferred site. 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > base.bind.ad Connected to domain:yourdomain.com server:yourdomain400.yourdomain.com in 0.011985 secs 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > base.bind.healing reset yourdomain.com(GC) disconnect state to connected 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > base.bind.cache pagedSearch :(&(keywords=Version:Centrify.Vegas.Installation.1)(keywords=InstallationId:fcacc433-33c4-4521-8540-23bd21064ffb)):S:0 attrs f4dca0f3 (useCache=1, GC=1) 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > base.bind.ldap init: Active searches 1 
 
Sep 20 12:49:13 fileserver adclient[2135]: DEBUG <fd:17 CAPILdapPagedSearch > base.objecthelper.ad cached object CN=SearchMark,CN=CENTRIFY MARKER,DC=$ attribute list (-186867469) didn't contain requested list f4dca0f3
 
Workaround:
None, other than restarting or running adflush.
 
Resolution:
Please contact Support for a special build which fixes this issue (CDC 5.1.1-833).
 
This is fixed in the upcoming release of Centrify DirectControl 5.1.2. 
 

Still have questions? Click here to log a technical support case, or collaborate with your peers in Centrify's Online Community.