Heads up:
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.
Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office for a Broad Institute event from Dec 10th to Dec 11th 2019. We will be back to monitor the GATK forum on Dec 12th 2019. In the meantime we encourage you to help out other community members with their queries.
Thank you for your patience!

ModelSegment error: Exception in thread "main" java.lang.OutOfMemoryError:GC overhead limit exceeded

Hi,

I am trying to follow the CNV somatic pipeline but I got stucked in ModelSegments, it seems that I do not give enough memory to the tool but it seems to me a little bit odd. I am actually giving it 30Gb and still the error persist:

gatk --java-options "-Xmx30g" ModelSegments ...

The error:

16:00:52.742 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/soft/EB_repo/bio/sequence/programs/foss/2016b/GATK/4.1.4.0/gatk-package-4.1.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
Nov 11, 2019 4:00:56 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
16:00:56.789 INFO  ModelSegments - ------------------------------------------------------------
16:00:56.790 INFO  ModelSegments - The Genome Analysis Toolkit (GATK) v4.1.4.0
16:00:56.790 INFO  ModelSegments - For support and documentation go to https://software.broadinstitute.org/gatk/
16:00:56.791 INFO  ModelSegments - Executing as [email protected] on Linux v3.10.0-862.14.4.el7.x86_64 amd64
16:00:56.792 INFO  ModelSegments - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_152-release-1056-b12
16:00:56.792 INFO  ModelSegments - Start Date/Time: November 11, 2019 4:00:52 PM CET
16:00:56.793 INFO  ModelSegments - ------------------------------------------------------------
16:00:56.793 INFO  ModelSegments - ------------------------------------------------------------
16:00:56.794 INFO  ModelSegments - HTSJDK Version: 2.20.3
16:00:56.794 INFO  ModelSegments - Picard Version: 2.21.1
16:00:56.795 INFO  ModelSegments - HTSJDK Defaults.COMPRESSION_LEVEL : 2
16:00:56.795 INFO  ModelSegments - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
16:00:56.795 INFO  ModelSegments - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
16:00:56.796 INFO  ModelSegments - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
16:00:56.796 INFO  ModelSegments - Deflater: IntelDeflater
16:00:56.796 INFO  ModelSegments - Inflater: IntelInflater
16:00:56.797 INFO  ModelSegments - GCS max retries/reopens: 20
16:00:56.797 INFO  ModelSegments - Requester pays: disabled
16:00:56.797 INFO  ModelSegments - Initializing engine
16:00:56.798 INFO  ModelSegments - Done initializing engine
16:00:56.903 INFO  ModelSegments - Reading file (../8504_hg38.bam.counts.hdf5.denoisedCR.tsv)...
16:00:59.370 INFO  ModelSegments - Reading file (../8504_hg38.bam.allelicCounts.tsv)...
16:22:25.491 INFO  ModelSegments - Reading file (../8503_hg38.bam.allelicCounts.tsv)...
18:36:31.973 INFO  ModelSegments - Shutting down engine
[November 11, 2019 6:36:31 PM CET] org.broadinstitute.hellbender.tools.copynumber.ModelSegments done. Elapsed time: 155.66 minutes.
Runtime.totalMemory()=28633464832
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
    at java.lang.StringBuilder.<init>(StringBuilder.java:101)
    at com.opencsv.CSVParser.parseLine(CSVParser.java:339)
    at com.opencsv.CSVParser.parseLineMulti(CSVParser.java:299)
    at com.opencsv.CSVReader.readNext(CSVReader.java:275)
    at org.broadinstitute.hellbender.utils.tsv.TableReader.fetchNextRecord(TableReader.java:355)
    at org.broadinstitute.hellbender.utils.tsv.TableReader.access$200(TableReader.java:97)
    at org.broadinstitute.hellbender.utils.tsv.TableReader$1.hasNext(TableReader.java:470)
    at java.util.Iterator.forEachRemaining(Iterator.java:115)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at org.broadinstitute.hellbender.tools.copynumber.formats.collections.AbstractRecordCollection.<init>(AbstractRecordCollection.java:84)
    at org.broadinstitute.hellbender.tools.copynumber.formats.collections.AbstractLocatableCollection.<init>(AbstractLocatableCollection.java:58)
    at org.broadinstitute.hellbender.tools.copynumber.formats.collections.AbstractSampleLocatableCollection.<init>(AbstractSampleLocatableCollection.java:44)
    at org.broadinstitute.hellbender.tools.copynumber.formats.collections.AllelicCountCollection.<init>(AllelicCountCollection.java:58)
    at org.broadinstitute.hellbender.tools.copynumber.ModelSegments$$Lambda$77/163015520.apply(Unknown Source)
    at org.broadinstitute.hellbender.tools.copynumber.ModelSegments.readOptionalFileOrNull(ModelSegments.java:599)
    at org.broadinstitute.hellbender.tools.copynumber.ModelSegments.doWork(ModelSegments.java:482)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206)
    at org.broadinstitute.hellbender.Main.main(Main.java:292)
Using GATK jar /soft/EB_repo/bio/sequence/programs/foss/2016b/GATK/4.1.4.0/gatk-package-4.1.4.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx30g -jar /soft/EB_repo/bio/sequence/programs/foss/2016b/GATK/4.1.4.0/gatk-package-4.1.4.0-local.jar ModelSegments --denoised-copy-ratios ../8504_hg38.bam.counts.hdf5.denoisedCR.tsv --allelic-counts ../8504_hg38.bam.allelicCounts.tsv --normal-allelic-counts ../8503_hg38.bam.allelicCounts.tsv --output result --output-prefix 8504

Any thoughts?

Thanks!

Best Answer

Answers

Sign In or Register to comment.