Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office on November 11th and 13th 2019, due to the U.S. holiday(Veteran's day) and due to a team event(Nov 13th). We will return to monitoring the GATK forum on November 12th and 14th respectively. Thank you for your patience.

Mutect2 error "Exception in thread "main" java.lang.OutOfMemoryError: Java heap space"

Running a Mutect2 using only the required arguments gives me OutOfMemoryError. The same happens with HaplotypeCaller. I run:

./gatk Mutect2 -R /hg19.fa -I /xx.bam -O /output.vcf.gz -tumor 01

Any thoughts what might be causing it?

Answers

  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin

    @KUBN

    Please post the version of gatk you are using and the entire error log.

  • KUBNKUBN Member
    edited May 4
    Thank you @bhanuGandham . Can I also add that I tried running with --java-options "-Xmx4g" and specifying the interval of 10kb (-L chr6:31130114-31140470), is it really possible and likely that I don't have enough memory to run this analysis?

    I tried --help, --version and --CountReads commands and all work fine.
    The version is 4.0.8.1-4 and the entire log is:
    ```
    Nadas-MacBook-Pro:gatk nadakubikova$ ./gatk HaplotypeCaller -R /Users/nadakubikova/Desktop/HumanGenome/hg19.fa -I /Users/nadakubikova/Downloads/wetransfer-7475fd/C13K_S21.bam -O /Users/nadakubikova/Desktop/HumanGenome/output.g.vcf.gz --java-options "-Xmx4g" -L chr6:31130114-31140470
    Using GATK jar /Users/nadakubikova/Fish/basic/gatk/build/libs/gatk-package-4.0.8.1-4-g1dbd042-SNAPSHOT-local.jar
    Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx4g -jar /Users/nadakubikova/Fish/basic/gatk/build/libs/gatk-package-4.0.8.1-4-g1dbd042-SNAPSHOT-local.jar HaplotypeCaller -R /Users/nadakubikova/Desktop/HumanGenome/hg19.fa -I /Users/nadakubikova/Downloads/wetransfer-7475fd/C13K_S21.bam -O /Users/nadakubikova/Desktop/HumanGenome/output.g.vcf.gz -L chr6:31130114-31140470
    14:56:17.074 INFO NativeLibraryLoader - Loading libgkl_compression.dylib from jar:file:/Users/nadakubikova/Fish/basic/gatk/build/libs/gatk-package-4.0.8.1-4-g1dbd042-SNAPSHOT-local.jar!/com/intel/gkl/native/libgkl_compression.dylib
    14:56:17.249 INFO HaplotypeCaller - ------------------------------------------------------------
    14:56:17.250 INFO HaplotypeCaller - The Genome Analysis Toolkit (GATK) v4.0.8.1-4-g1dbd042-SNAPSHOT
    14:56:17.250 INFO HaplotypeCaller - For support and documentation go to
    14:56:17.250 INFO HaplotypeCaller - Executing as [email protected] on Mac OS X v10.14.4 x86_64
    14:56:17.250 INFO HaplotypeCaller - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_181-b13
    14:56:17.250 INFO HaplotypeCaller - Start Date/Time: 04 May 2019 14:56:17 BST
    14:56:17.250 INFO HaplotypeCaller - ------------------------------------------------------------
    14:56:17.250 INFO HaplotypeCaller - ------------------------------------------------------------
    14:56:17.251 INFO HaplotypeCaller - HTSJDK Version: 2.16.0
    14:56:17.251 INFO HaplotypeCaller - Picard Version: 2.18.7
    14:56:17.251 INFO HaplotypeCaller - HTSJDK Defaults.COMPRESSION_LEVEL : 2
    14:56:17.251 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    14:56:17.251 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    14:56:17.251 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    14:56:17.251 INFO HaplotypeCaller - Deflater: IntelDeflater
    14:56:17.251 INFO HaplotypeCaller - Inflater: IntelInflater
    14:56:17.251 INFO HaplotypeCaller - GCS max retries/reopens: 20
    14:56:17.251 INFO HaplotypeCaller - Using google-cloud-java fork github.com/broadinstitute/google-cloud-java/releases/tag/0.20.5-alpha-GCS-RETRY-FIX
    14:56:17.251 INFO HaplotypeCaller - Initializing engine
    15:00:21.031 INFO HaplotypeCaller - Shutting down engine
    [04 May 2019 15:00:21 BST] org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller done. Elapsed time: 4.07 minutes.
    Runtime.totalMemory()=3817865216
    Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
    at htsjdk.samtools.SAMTextHeaderCodec$ParsedHeaderLine.<init>(SAMTextHeaderCodec.java:287)
    at htsjdk.samtools.SAMTextHeaderCodec.decode(SAMTextHeaderCodec.java:95)
    at htsjdk.samtools.reference.ReferenceSequenceFileFactory.loadDictionary(ReferenceSequenceFileFactory.java:232)
    at htsjdk.samtools.reference.AbstractFastaSequenceFile.<init>(AbstractFastaSequenceFile.java:68)
    at htsjdk.samtools.reference.AbstractIndexedFastaSequenceFile.<init>(AbstractIndexedFastaSequenceFile.java:60)
    at htsjdk.samtools.reference.IndexedFastaSequenceFile.<init>(IndexedFastaSequenceFile.java:80)
    at htsjdk.samtools.reference.IndexedFastaSequenceFile.<init>(IndexedFastaSequenceFile.java:98)
    at org.broadinstitute.hellbender.utils.fasta.CachingIndexedFastaSequenceFile.<init>(CachingIndexedFastaSequenceFile.java:98)
    at org.broadinstitute.hellbender.utils.fasta.CachingIndexedFastaSequenceFile.checkAndCreate(CachingIndexedFastaSequenceFile.java:205)
    at org.broadinstitute.hellbender.utils.fasta.CachingIndexedFastaSequenceFile.checkAndCreate(CachingIndexedFastaSequenceFile.java:183)
    at org.broadinstitute.hellbender.engine.ReferenceFileSource.<init>(ReferenceFileSource.java:37)
    at org.broadinstitute.hellbender.engine.ReferenceDataSource.of(ReferenceDataSource.java:27)
    at org.broadinstitute.hellbender.engine.GATKTool.initializeReference(GATKTool.java:364)
    at org.broadinstitute.hellbender.engine.GATKTool.onStartup(GATKTool.java:634)
    at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.onStartup(AssemblyRegionWalker.java:156)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:135)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:182)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:201)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)
    Nadas-MacBook-Pro:gatk nadakubikova$
    ```

    Any insights would be greatly appreciated!

    Best wishes,
    Nada
  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin

    @KUBN

    How much physical memory do you have on the machine you are running this analysis? GATK tools need about 2-3GB of memory for native libraries in addition to the -Xmx value you set.

  • KUBNKUBN Member
    @bhanuGandham

    Thanks for your comment, I have got in total 8GB of physical memory.
  • KUBNKUBN Member
    @bhanuGandham

    I made sure to free up 5GB of memory. However, I have no luck running the the analysis and I still get the same error code, matter the -Xmx value I set. The bam file is not a large one. Shouldn't specifying the interval speed up the analysis?
  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin
    edited May 7

    Hi @KUBN

    I see you have posted the error log for HaplotypeCaller. Would you please post the error log for the Mutect2 command.
    Also, I would also like to know what the depth is like and whether it's particularly high where the error occurs.

    Post edited by bhanuGandham on
Sign In or Register to comment.