Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.

Mutect2 out of memory - not enough java heap space

jejacobs23jejacobs23 Portland, ORMember

I am running Mutect2 on whole genome sequencing and have run into an error with java. It seems to be running out of heap space towards the end of the run. I plan to add more with the -Xmx option and also spread the jobs out with intervals. However, I am wondering if there is a way to predict or determine how much heap space is actually needed for a Mutect2 run.

Best Answer

Answers

  • SChaluvadiSChaluvadi Member, Broadie, Moderator admin

    @jejacobs23
    GATK tools need about 2-3GB of memory for native libraries in addition to the -Xmx value you set. The additional information you might need to use to determine memory is size of the input files and possibly read depth etc. Unfortunately we do not have official recommendations for setting the amount of memory since it requires some experimenting based on each individual input. Spreading over intervals however should help the java heap space issue.

  • jejacobs23jejacobs23 Portland, ORMember

    Thanks. That makes sense. Below, I've included the error log from my Mutect2 run with "-Xmx4g".

    10:29:35.106 INFO  VectorLoglessPairHMM - Time spent in setup for JNI call : 138.42475344800002
    10:29:35.106 INFO  PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 14546.248884447
    10:29:35.107 INFO  SmithWatermanAligner - Total compute time in java Smith-Waterman : 21646.26 sec
    INFO    2019-05-18 10:30:05     SortingCollection       Creating merging iterator from 76 files
    10:32:47.823 INFO  Mutect2 - Shutting down engine
    [May 18, 2019 10:32:47 AM PDT] org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2 done. Elapsed time: 2,662.33 minutes.
    Runtime.totalMemory()=4120379392
    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar replaced objects
            at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
            at java.lang.StringBuilder.<init>(StringBuilder.java:89)
            at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.readthreading.ReadThreadingGraph.buildGraphIfNecessary(ReadThreadingGraph.java:340)
            at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.readthreading.ReadThreadingAssembler.createGraph(ReadThreadingAssembler.java:414)
            at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.readthreading.ReadThreadingAssembler.assemble(ReadThreadingAssembler.java:358)
            at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.readthreading.ReadThreadingAssembler.runLocalAssembly(ReadThreadingAssembler.java:142)
            at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.AssemblyBasedCallerUtils.assembleReads(AssemblyBasedCallerUtils.java:288)
            at org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2Engine.callRegion(Mutect2Engine.java:225)
            at org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2.apply(Mutect2.java:324)
            at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.processReadShard(AssemblyRegionWalker.java:308)
            at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.traverse(AssemblyRegionWalker.java:281)
            at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1039)
            at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
            at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
            at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
            at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
            at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
            at org.broadinstitute.hellbender.Main.main(Main.java:291)
    

    Just curious, does the Runtime.totalMemory() = 4120379392 tell me anything about how much heap space I need?

Sign In or Register to comment.