The Frontline Support team will be slow to respond December 17-18 due to an institute-wide retreat and offline December 22- January 1, while the institute is closed. Thank you for your patience during these next few weeks. Happy Holidays!
Crashes with segmentation fault in shipped `libVectorLoglessPairHMM.so`
Dear GATK people,
Our scientists reported, that GATK 3.6 and 3.7 terminates with a segmentation fault on a lot of Intel systems. Crazily enough, it’s not reproducible on all of them despite being the same model and operating system.
# # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f6c2fa64ce9, pid=4189, tid=0x00007fa8dfc34700 # # JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build 1.8.0_131-b11) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 ) # Problematic frame: # C [libVectorLoglessPairHMM9026850853863068944.so+0x1bce9] LoadTimeInitializer::LoadTimeInitializer()+0x1669 # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # An error report file with more information is saved as: # /scratch/local/joey/workdir/hs_err_pid4189.log # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. #
It also happens with older Java versions.
# JRE version: Java(TM) SE Runtime Environment (8.0_25-b17) (build 1.8.0_25-b17) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.25-b02 mixed mode linux-amd64 )
It always terminates at the same address. Here is the top of the error report.
[…] --------------- T H R E A D --------------- Current thread (0x00007fa8d8009800): JavaThread "main" [_thread_in_native, id=4190, stack(0x00007fa8dfb34000,0x00007fa8dfc35000)] siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007f6a2fd91680 Registers: RAX=0xffffffff80000000, RBX=0x0000000000000000, RCX=0x0000000000000000, RDX=0x0000000000000001 RSP=0x00007fa8dfc319f0, RBP=0x0000000000000000, RSI=0x00007f6c2fddf8a0, RDI=0x0000000000000000 R8 =0x00007f6c2fa90c80, R9 =0x00007f6c2fa90dd0, R10=0x000000000000009c, R11=0x00007f6c2fa69450 R12=0x0000000000000002, R13=0x00007f6c2fd91680, R14=0x00000000000138ff, R15=0x00007f6c2fddf8a4 RIP=0x00007f6c2fa64ce9, EFLAGS=0x0000000000010247, CSGSFS=0x002b000000000033, ERR=0x0000000000000004 TRAPNO=0x000000000000000e […]
Here is the backtrace from GDB on the core dump file.
$ gdb java /scratch/local/joey/workdir/core […] (gdb) bt #0 __GI_raise ([email protected]=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007f486d6c275a in __GI_abort () at abort.c:89 #2 0x00007f486cfc13b5 in os::abort(bool) () from /usr/local/java/jre/lib/amd64/server/libjvm.so #3 0x00007f486d163673 in VMError::report_and_die() () from /usr/local/java/jre/lib/amd64/server/libjvm.so #4 0x00007f486cfc68bf in JVM_handle_linux_signal () from /usr/local/java/jre/lib/amd64/server/libjvm.so #5 0x00007f486cfbce13 in signalHandler(int, siginfo*, void*) () from /usr/local/java/jre/lib/amd64/server/libjvm.so #6 <signal handler called> #7 0x00007f0bd6245ce9 in LoadTimeInitializer::LoadTimeInitializer() () from /project/seqcore-cluster/temp/joseph/libVectorLoglessPairHMM1517996528329513895.so #8 0x00007f0bd6243487 in __sti__$E () from /project/seqcore-cluster/temp/joseph/libVectorLoglessPairHMM1517996528329513895.so #9 0x00007f0bd625f126 in __do_global_ctors_aux () from /project/seqcore-cluster/temp/joseph/libVectorLoglessPairHMM1517996528329513895.so #10 0x00007f0bd62326fb in _init () from /project/seqcore-cluster/temp/joseph/libVectorLoglessPairHMM1517996528329513895.so […]
Interestingly, when rebuilding the library with GCC 5.3, and when loading our library with the documented option below, it fixes the issue for us.
PS: Hopefully, this is the correct forum for such a report. The GitHub repository does not have the issue tracker enabled.