Centrify DirectControl 5.2.2 and DirectAudit 3.2.2 (Enterprise Suite 2015) and lower version on All Unix OS Platforms
Problem:
Customer may encounter JVM failures after installing and enabling Centrify DirectAudit 3.2.2-338.
Below is the output of the error you may see: # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f5e6daa12fe, pid=11386, tid=140043123902208 # # JRE version: 7.0_25-b15 # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.25-b01 mixed mode linux-amd64 compressed oops) # Problematic frame: # C [libnss_centrifyda.so.2+0x112fe] SessionSendWithTimeout+0x16e #
Error Message: Illegal memory access. [54] Signal info : si_signo=11, si_code=1 si_addr=0x26 Version : Oracle JRockit(R) R28.1.3-11-141760-1.6.0_24-20110301-1432-linux-x86_64 CPU : Intel (null) (HT) SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 Intel64 Number CPUs : 40 Tot Phys Mem : 135121494016 (128861 MB) OS version : Red Hat Enterprise Linux Server release 5.11 (Tikanga) Linux version 2.6.18-406.el5 (mockbuild@x86-026.build.eng.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-55)) #1 SMP Fri May 1 10:37:57 EDT 2015 (x86_64) Thread System: Linux NPTL LibC release : 2.5-stable Java locking : Lazy unlocking enabled (class banning) (transfer banning) State : JVM is running
Customer may also see core dump file with SessionSendWithTimeout error :
0x00002ba3fa9b6e17 in VMError::report_and_die() () from /app/bmc/BladeLogic/8.1/NSH/br/java/lib/amd64/server/libjvm.so 0x00002ba3fa83a49f in JVM_handle_linux_signal () from /app/bmc/BladeLogic/8.1/NSH/br/java/lib/amd64/server/libjvm.so <signal handler called> 0x00002ba3fb3db4fe in SessionSendWithTimeout () from /lib64/libnss_centrifyda.so.2 0x00002ba3fb3d3b48 in SendMessage () from /lib64/libnss_centrifyda.so.2 0x00002ba3fb3d3d46 in nSSQueryDaemonByUid () from /lib64/libnss_centrifyda.so.2 0x00002ba3fb3d3e37 in nSSQueryDaemonByUid_o () from /lib64/libnss_centrifyda.so.2
If Centrify DirectAudit is uninstalled or the service is disabled, the Java process works fine.
Cause:
Prior to Suite 2015.1, the Direct Audit NSS library uses the system call select() to process its interprocess communication with the DirectAudit daemon (dad). The select() call requires a bit mask of file descriptors to check. However, the system macro that we use to set the bit mask does not work correctly where the file descriptor is a large number (which is a possible case for applications that opens a lot of files/sockets, such as JVM), leading to unpredictable memory corruption. We fix this in Suite 2015.1 by re-implementing the interprocess communication without using the select() call.
Workaround:
enable nscd
Solution:
Upgrade to Enterprise Centrify Suite 2015.1 that has DirectAudit package included (CDC 5.2.3 with CDA 3.2.3).
