Download the latest Picard release at https://github.com/broadinstitute/picard/releases.
GATK version 4.beta.5 is out. See the GATK4 beta page for download and details.

Crashes with segmentation fault in shipped `libVectorLoglessPairHMM.so`

Dear GATK people,

Our scientists reported, that GATK 3.6 and 3.7 terminates with a segmentation fault on a lot of Intel systems. Crazily enough, it’s not reproducible on all of them despite being the same model and operating system.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f6c2fa64ce9, pid=4189, tid=0x00007fa8dfc34700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_131-b11) (build 1.8.0_131-b11)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.131-b11 mixed mode linux-amd64 )
# Problematic frame:
# C  [libVectorLoglessPairHMM9026850853863068944.so+0x1bce9]  LoadTimeInitializer::LoadTimeInitializer()+0x1669
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /scratch/local/joey/workdir/hs_err_pid4189.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

It also happens with older Java versions.

# JRE version: Java(TM) SE Runtime Environment (8.0_25-b17) (build 1.8.0_25-b17)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.25-b02 mixed mode linux-amd64 )

It always terminates at the same address. Here is the top of the error report.

[…]
---------------  T H R E A D  ---------------

Current thread (0x00007fa8d8009800):  JavaThread "main" [_thread_in_native, id=4190, stack(0x00007fa8dfb34000,0x00007fa8dfc35000)]

siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007f6a2fd91680

Registers:
RAX=0xffffffff80000000, RBX=0x0000000000000000, RCX=0x0000000000000000, RDX=0x0000000000000001
RSP=0x00007fa8dfc319f0, RBP=0x0000000000000000, RSI=0x00007f6c2fddf8a0, RDI=0x0000000000000000
R8 =0x00007f6c2fa90c80, R9 =0x00007f6c2fa90dd0, R10=0x000000000000009c, R11=0x00007f6c2fa69450
R12=0x0000000000000002, R13=0x00007f6c2fd91680, R14=0x00000000000138ff, R15=0x00007f6c2fddf8a4
RIP=0x00007f6c2fa64ce9, EFLAGS=0x0000000000010247, CSGSFS=0x002b000000000033, ERR=0x0000000000000004
  TRAPNO=0x000000000000000e
[…]

Here is the backtrace from GDB on the core dump file.

$ gdb java /scratch/local/joey/workdir/core
[…]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007f486d6c275a in __GI_abort () at abort.c:89
#2  0x00007f486cfc13b5 in os::abort(bool) () from /usr/local/java/jre/lib/amd64/server/libjvm.so
#3  0x00007f486d163673 in VMError::report_and_die() () from /usr/local/java/jre/lib/amd64/server/libjvm.so
#4  0x00007f486cfc68bf in JVM_handle_linux_signal () from /usr/local/java/jre/lib/amd64/server/libjvm.so
#5  0x00007f486cfbce13 in signalHandler(int, siginfo*, void*) () from /usr/local/java/jre/lib/amd64/server/libjvm.so
#6  <signal handler called>
#7  0x00007f0bd6245ce9 in LoadTimeInitializer::LoadTimeInitializer() () from /project/seqcore-cluster/temp/joseph/libVectorLoglessPairHMM1517996528329513895.so
#8  0x00007f0bd6243487 in __sti__$E () from /project/seqcore-cluster/temp/joseph/libVectorLoglessPairHMM1517996528329513895.so
#9  0x00007f0bd625f126 in __do_global_ctors_aux () from /project/seqcore-cluster/temp/joseph/libVectorLoglessPairHMM1517996528329513895.so
#10 0x00007f0bd62326fb in _init () from /project/seqcore-cluster/temp/joseph/libVectorLoglessPairHMM1517996528329513895.so
[…]

Interestingly, when rebuilding the library with GCC 5.3, and when loading our library with the documented option below, it fixes the issue for us.

-Djava.library.path=/src/gatk/public/VectorPairHMM/src/main/c++

PS: Hopefully, this is the correct forum for such a report. The GitHub repository does not have the issue tracker enabled.

Tagged:

Issue · Github
by Geraldine_VdAuwera

Issue Number
2270
State
closed
Last Updated
Assignee
Array
Milestone
Array
Closed By
vdauwera

Answers

Sign In or Register to comment.