We've moved!
This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!

ModelSegment error: Exception in thread "main" java.lang.OutOfMemoryError:GC overhead limit exceeded

Hi,

I am trying to follow the CNV somatic pipeline but I got stucked in ModelSegments, it seems that I do not give enough memory to the tool but it seems to me a little bit odd. I am actually giving it 30Gb and still the error persist:

gatk --java-options "-Xmx30g" ModelSegments ...

The error:

16:00:52.742 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/soft/EB_repo/bio/sequence/programs/foss/2016b/GATK/4.1.4.0/gatk-package-4.1.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
Nov 11, 2019 4:00:56 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
16:00:56.789 INFO  ModelSegments - ------------------------------------------------------------
16:00:56.790 INFO  ModelSegments - The Genome Analysis Toolkit (GATK) v4.1.4.0
16:00:56.790 INFO  ModelSegments - For support and documentation go to https://software.broadinstitute.org/gatk/
16:00:56.791 INFO  ModelSegments - Executing as [email protected] on Linux v3.10.0-862.14.4.el7.x86_64 amd64
16:00:56.792 INFO  ModelSegments - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_152-release-1056-b12
16:00:56.792 INFO  ModelSegments - Start Date/Time: November 11, 2019 4:00:52 PM CET
16:00:56.793 INFO  ModelSegments - ------------------------------------------------------------
16:00:56.793 INFO  ModelSegments - ------------------------------------------------------------
16:00:56.794 INFO  ModelSegments - HTSJDK Version: 2.20.3
16:00:56.794 INFO  ModelSegments - Picard Version: 2.21.1
16:00:56.795 INFO  ModelSegments - HTSJDK Defaults.COMPRESSION_LEVEL : 2
16:00:56.795 INFO  ModelSegments - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
16:00:56.795 INFO  ModelSegments - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
16:00:56.796 INFO  ModelSegments - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
16:00:56.796 INFO  ModelSegments - Deflater: IntelDeflater
16:00:56.796 INFO  ModelSegments - Inflater: IntelInflater
16:00:56.797 INFO  ModelSegments - GCS max retries/reopens: 20
16:00:56.797 INFO  ModelSegments - Requester pays: disabled
16:00:56.797 INFO  ModelSegments - Initializing engine
16:00:56.798 INFO  ModelSegments - Done initializing engine
16:00:56.903 INFO  ModelSegments - Reading file (../8504_hg38.bam.counts.hdf5.denoisedCR.tsv)...
16:00:59.370 INFO  ModelSegments - Reading file (../8504_hg38.bam.allelicCounts.tsv)...
16:22:25.491 INFO  ModelSegments - Reading file (../8503_hg38.bam.allelicCounts.tsv)...
18:36:31.973 INFO  ModelSegments - Shutting down engine
[November 11, 2019 6:36:31 PM CET] org.broadinstitute.hellbender.tools.copynumber.ModelSegments done. Elapsed time: 155.66 minutes.
Runtime.totalMemory()=28633464832
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
    at java.lang.StringBuilder.<init>(StringBuilder.java:101)
    at com.opencsv.CSVParser.parseLine(CSVParser.java:339)
    at com.opencsv.CSVParser.parseLineMulti(CSVParser.java:299)
    at com.opencsv.CSVReader.readNext(CSVReader.java:275)
    at org.broadinstitute.hellbender.utils.tsv.TableReader.fetchNextRecord(TableReader.java:355)
    at org.broadinstitute.hellbender.utils.tsv.TableReader.access$200(TableReader.java:97)
    at org.broadinstitute.hellbender.utils.tsv.TableReader$1.hasNext(TableReader.java:470)
    at java.util.Iterator.forEachRemaining(Iterator.java:115)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at org.broadinstitute.hellbender.tools.copynumber.formats.collections.AbstractRecordCollection.<init>(AbstractRecordCollection.java:84)
    at org.broadinstitute.hellbender.tools.copynumber.formats.collections.AbstractLocatableCollection.<init>(AbstractLocatableCollection.java:58)
    at org.broadinstitute.hellbender.tools.copynumber.formats.collections.AbstractSampleLocatableCollection.<init>(AbstractSampleLocatableCollection.java:44)
    at org.broadinstitute.hellbender.tools.copynumber.formats.collections.AllelicCountCollection.<init>(AllelicCountCollection.java:58)
    at org.broadinstitute.hellbender.tools.copynumber.ModelSegments$$Lambda$77/163015520.apply(Unknown Source)
    at org.broadinstitute.hellbender.tools.copynumber.ModelSegments.readOptionalFileOrNull(ModelSegments.java:599)
    at org.broadinstitute.hellbender.tools.copynumber.ModelSegments.doWork(ModelSegments.java:482)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206)
    at org.broadinstitute.hellbender.Main.main(Main.java:292)
Using GATK jar /soft/EB_repo/bio/sequence/programs/foss/2016b/GATK/4.1.4.0/gatk-package-4.1.4.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx30g -jar /soft/EB_repo/bio/sequence/programs/foss/2016b/GATK/4.1.4.0/gatk-package-4.1.4.0-local.jar ModelSegments --denoised-copy-ratios ../8504_hg38.bam.counts.hdf5.denoisedCR.tsv --allelic-counts ../8504_hg38.bam.allelicCounts.tsv --normal-allelic-counts ../8503_hg38.bam.allelicCounts.tsv --output result --output-prefix 8504

Any thoughts?

Thanks!

Best Answer

Answers

Sign In or Register to comment.