Bug Bulletin: The recent 3.2 release fixes many issues. If you run into a problem, please try the latest version before posting a bug report, as your problem may already have been solved.

UnifiedGenotyper core dump

kostjkostj Posts: 1Member
edited March 2013 in Ask the GATK team

While I have been able to work around this issue, I am still curious as to what is causing it in the first place.

I am currently engaged in variant calling on a rather large set of exome sequenced samples (prepared according to GATK best practices 4), but I have run into some issues with UnifiedGenotyper.

Put simply, I am able to run any of our input intervals through UnfiedGenotyper with a single data thread, and either 1 or 24 CPU threads with no trouble. Upon looking at the actual memory usage of the process while running, I decided to try increasing the number of data threads as it appeared that I could easily run 4 threads in the heap I have available. However, regardless of the thread or memory settings, when I raise the number of data threads beyond 1, once the initialization phase has completed, I immediately get a Java core dump. The GATK invocation I use is as follows:

java -Djava.io.tmpdir=$TMP -Xmx48G -jar $GATK -T UnifiedGenotyper \
            -R $REF \
            --dbsnp $ROD \
            ${GATKArg} \
            -o ${TMP}/${BASENAME}/${BASENAME}.${INTERVAL}.raw.variants.vcf \
            -L ${INTERVAL} \
                            -nt 4 \
            -nct 8 \
            -glm BOTH \
            -A DepthOfCoverage \
            -A AlleleBalance \
            -A HaplotypeScore \
            -A HomopolymerRun \
            -A MappingQualityZero \
            -A QualByDepth \
            -A RMSMappingQuality \
            -A SpanningDeletions \
            -A MappingQualityRankSumTest \
            -A ReadPosRankSumTest \
            -A FisherStrand \
            -A InbreedingCoeff

For reference, this is GATK version 2.3-9, JRE 1.6.

The error I receive is shown below. Any insight into this issue would be greatly appreciated.

Sincerely, Jason Kost

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (exceptions.cpp:364), pid=14830, tid=1113614656
#  Error: ExceptionMark destructor expects no pending exceptions
#
# JRE version: 6.0_16-b01
# Java VM: Java HotSpot(TM) 64-Bit Server VM (14.2-b01 mixed mode linux-amd64 )
# Can not save log file, dump to screen..
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (exceptions.cpp:364), pid=14830, tid=1113614656
#  Error: ExceptionMark destructor expects no pending exceptions
#
# JRE version: 6.0_16-b01
# Java VM: Java HotSpot(TM) 64-Bit Server VM (14.2-b01 mixed mode linux-amd64 )
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#

---------------  T H R E A D  ---------------

Current thread (0x00000000490c3800):  JavaThread "S3Put-Thread" daemon [_thread_in_vm, id=14864, stack(0x0000000042506000,0x0000000042607000)]

Stack: [0x0000000042506000,0x0000000042607000],  sp=0x0000000042602bd0,  free space=1010k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x6bd1ef]
V  [libjvm.so+0x2be556]
V  [libjvm.so+0x3082dd]
V  [libjvm.so+0x2592d0]
V  [libjvm.so+0x258922]
V  [libjvm.so+0x2589a6]
V  [libjvm.so+0x25a37e]
V  [libjvm.so+0x650eab]
V  [libjvm.so+0x64f222]
V  [libjvm.so+0x64e101]
V  [libjvm.so+0x64dd63]
V  [libjvm.so+0x42c8e9]
V  [libjvm.so+0x43310e]
V  [libjvm.so+0x411d68]

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.lang.ClassLoader.findBootstrapClass(Ljava/lang/String;)Ljava/lang/Class;+0
j  java.lang.ClassLoader.findBootstrapClass0(Ljava/lang/String;)Ljava/lang/Class;+23
j  java.lang.ClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;+32
j  java.lang.ClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;+23
j  sun.misc.Launcher$AppClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;+41
j  java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class;+3
j  java.lang.ClassLoader.loadClassInternal(Ljava/lang/String;)Ljava/lang/Class;+2
v  ~StubRoutines::call_stub
j  org.broadinstitute.sting.gatk.phonehome.GATKRunReport$S3PutRunnable.run()V+6
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
=>0x00000000490c3800 JavaThread "S3Put-Thread" daemon [_thread_in_vm, id=14864, stack(0x0000000042506000,0x0000000042607000)]
  0x0000000049207000 JavaThread "Thread-3" [_thread_in_native, id=14863, stack(0x0000000042405000,0x0000000042506000)]
  0x0000000049384800 JavaThread "Thread-2" [_thread_blocked, id=14862, stack(0x0000000042304000,0x0000000042405000)]
  0x00000000493fe000 JavaThread "ProgressMeterDaemon" daemon [_thread_blocked, id=14858, stack(0x0000000042203000,0x0000000042304000)]
  0x00002ab6c0026000 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=14856, stack(0x0000000042001000,0x0000000042102000)]
  0x00002ab6c0023000 JavaThread "CompilerThread1" daemon [_thread_blocked, id=14855, stack(0x0000000041bd6000,0x0000000041cd7000)]
  0x00002ab6c0020800 JavaThread "CompilerThread0" daemon [_thread_blocked, id=14854, stack(0x0000000041ad5000,0x0000000041bd6000)]
  0x00002ab6c001e800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=14853, stack(0x00000000419d4000,0x0000000041ad5000)]
  0x00000000484b3000 JavaThread "Finalizer" daemon [_thread_blocked, id=14852, stack(0x0000000041f00000,0x0000000042001000)]
  0x00000000484ab800 JavaThread "Reference Handler" daemon [_thread_blocked, id=14851, stack(0x0000000041dff000,0x0000000041f00000)]
  0x00000000481a8800 JavaThread "main" [_thread_blocked, id=14831, stack(0x0000000040450000,0x0000000040551000)]

Other Threads:
  0x00000000484a6800 VMThread [stack: 0x0000000041cfe000,0x0000000041dff000] [id=14850]
  0x00002ab6c0029000 WatcherThread [stack: 0x0000000042102000,0x0000000042203000] [id=14857]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 PSYoungGen      total 63360K, used 3359K [0x00002ab2b3440000, 0x00002ab2ba990000, 0x00002ab6b3440000)
  eden space 16384K, 20% used [0x00002ab2b3440000,0x00002ab2b3787fb8,0x00002ab2b4440000)
  from space 46976K, 0% used [0x00002ab2b7550000,0x00002ab2b7550000,0x00002ab2ba330000)
  to   space 50240K, 0% used [0x00002ab2b4440000,0x00002ab2b4440000,0x00002ab2b7550000)
 PSOldGen        total 353536K, used 271234K [0x00002aaab3440000, 0x00002aaac8d80000, 0x00002ab2b3440000)
  object space 353536K, 76% used [0x00002aaab3440000,0x00002aaac3d20bd8,0x00002aaac8d80000)
 PSPermGen       total 21504K, used 16865K [0x00002aaaae040000, 0x00002aaaaf540000, 0x00002aaab3440000)
  object space 21504K, 78% used [0x00002aaaae040000,0x00002aaaaf0b8538,0x00002aaaaf540000)

Dynamic libraries:
Can not get library information for pid = 14864

VM Arguments:
jvm_args: -Djava.io.tmpdir=/home/kostj/scratch/temp/ -Xmx48G 
java_command: /home/kostj/nearline/gatk2/GenomeAnalysisTK.jar -T UnifiedGenotyper -R /home/kostj/nearline/hg19/hg19.fasta --dbsnp /home/kostj/nearline/hg19/gatk2/dbsnp_137.hg19.vcf -I /home/kostj/scratch/temp//SPr0059/SPr0059.reduced.bam -I /home/kostj/scratch/temp//SP3608/SP3608.reduced.bam -I /home/kostj/scratch/temp//SP3528/SP3528.reduced.bam -I /home/kostj/scratch/temp//SP3508/SP3508.reduced.bam -I /home/kostj/scratch/temp//SP3474/SP3474.reduced.bam -I /home/kostj/scratch/temp//SP3277/SP3277.reduced.bam -I /home/kostj/scratch/temp//SP3255/SP3255.reduced.bam -I /home/kostj/scratch/temp//SP3148/SP3148.reduced.bam -I /home/kostj/scratch/temp//sp3127/sp3127.reduced.bam -I /home/kostj/scratch/temp//SP3070/SP3070.reduced.bam -I /home/kostj/scratch/temp//sp3035/sp3035.reduced.bam -I /home/kostj/scratch/temp//SP0618/SP0618.reduced.bam -I /home/kostj/scratch/temp//SNc0063/SNc0063.reduced.bam -I /home/kostj/scratch/temp//SNc0036/SNc0036.reduced.bam -I /home/kostj/scratch/temp//SMa0078/SMa0078.reduced.bam -I /home/kostj/scratch/temp//SMa0020/SMa0020.reduced.bam -I /home/kostj/scratch/temp//SLA969/SLA969.reduced.bam -I /home/kostj/scratch/temp//SLA966/SLA966.reduced.bam -I /home/kostj/scratch/temp//SLA956/SLA956.reduced.bam -I /home/kostj/scratch/temp//SLA929/SLA929.reduced.bam -I /home/kostj/scratch/temp//SLA922/SLA922.reduced.bam -I /home/kostj/scratch/temp//SLA885/SLA885.reduced.bam -I /home/kostj/scratch/temp//SLA877/SLA877.reduced.bam -I /home/kostj/scratch/temp//SLA863/SLA863.reduced.bam -I /home/kostj/scratch/temp//SLA861/SLA861.reduced.bam -I /home/kostj/scratch/temp//SLA833/SLA833.reduced.bam -I /home/kostj/scratch/temp//SLA809/SLA809.reduced.bam -I /home/kostj/scratch/temp//SLA808/SLA808.reduced.bam -I /home/kostj/scratch/temp//SLA807/SLA807.reduced.bam -I /home/kostj/scratch/temp//SLA798/SLA798.reduced.bam -I /home/kostj/scratch/temp//SLA791/SLA791.reduced.bam -I /home/kostj/scratch/temp//SLA783/SLA783.reduced.bam -I /home/kostj/scratch/temp//SLA767/SLA767.redu
Launcher Type: SUN_STANDARD

Environment Variables:
PATH=/share/bin/java/bin:/home/kostj/nearline/R/bin/:/tmp/8474173.1.chassis12.q:/share/bin/gcc/bin:/share/bin:/opt/SUNWhpc/HPC8.1/gnu/bin/:/share/bin/intel/bin/intel64:/share/bin/gmp/include:/share/bin/mpfr/include:/sge/bin/lx24-amd64:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin
LD_LIBRARY_PATH=/share/bin/jre1.6.0_16/lib/amd64/server:/share/bin/jre1.6.0_16/lib/amd64:/share/bin/jre1.6.0_16/../lib/amd64:/lib:/usr/lib:/usr/local/lib:/opt/SUNWhpc/HPC8.1/gnu/lib/lib64:/share/bin/gcc/lib64:/share/bin/gcc/lib:/share/bin/intel/fc/10.1.015/lib:/share/bin/intel/fc/10.1.015/licenses:/share/bin/intel/cc/10.1.015/lib:/share/bin/mpfr/lib:/share/bin/gmp/lib:/share/lib:
SHELL=/bin/bash

Signal Handlers:
SIGSEGV: [libjvm.so+0x6bddc0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGBUS: [libjvm.so+0x6bddc0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGFPE: [libjvm.so+0x594f90], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGPIPE: [libjvm.so+0x594f90], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGXFSZ: [libjvm.so+0x594f90], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGILL: [libjvm.so+0x594f90], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: [libjvm.so+0x597750], sa_mask[0]=0x00000000, sa_flags=0x10000004
SIGHUP: [libjvm.so+0x5974a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGINT: [libjvm.so+0x5974a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGTERM: [libjvm.so+0x5974a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGQUIT: [libjvm.so+0x5974a0], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004


---------------  S Y S T E M  ---------------

OS:Linux
uname:Linux 2.6.18-92.el5 #1 SMP Tue Jun 10 18:51:06 EDT 2008 x86_64
libc:glibc 2.5 NPTL 2.5 
rlimit: STACK infinity, CORE infinity, NPROC 399360, NOFILE 1024, AS infinity
load average:0.00 0.00 0.00

CPU:total 24 (16 cores per cpu, 2 threads per core) family 6 model 44 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, ht

Memory: 4k page, physical 49451764k(23657140k free), swap 16779884k(16697572k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (14.2-b01) for linux-amd64 JRE (1.6.0_16-b01), built on Jul 31 2009 05:52:33 by "java_re" with gcc 3.2.2 (SuSE Linux)

time: Mon Mar 11 16:00:00 2013
elapsed time: 923 seconds

#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
#
/sge/default/spool/pem610-040/job_scripts/8474173: line 83: 14830 Aborted                 (core dumped) java -Djava.io.tmpdir=$TMP -Xmx48G -jar $GATK -T UnifiedGenotyper -R $REF --dbsnp $ROD ${GATKArg} -o ${TMP}/${BASENAME}/${BASENAME}.${INTERVAL}.raw.variants.vcf -L ${INTERVAL} -nt 2 -nct 12 -glm BOTH -A DepthOfCoverage -A AlleleBalance -A HaplotypeScore -A HomopolymerRun -A MappingQualityZero -A QualByDepth -A RMSMappingQuality -A SpanningDeletions -A MappingQualityRankSumTest -A ReadPosRankSumTest -A FisherStrand -A InbreedingCoeff
Post edited by Geraldine_VdAuwera on

Answers

Sign In or Register to comment.