Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office on October 14, 2019, due to the U.S. holiday. We will return to monitoring the forum on October 15.

GATK runtime error (READ_MAX_LENGTH must be > 0 but got 0) with 1000g bam

RaphaelHRaphaelH BrusselsMember

Hi,

I'm trying to build a pon with GATK 3.7-0 to use with mutect2. For that, I've downloaded 80 exome bam files from the 1000g project (GBR, TSI, IBS and CEU populations).
For most of them, when I try to use the artifact_dectection_mode, I get a GATK runtime error saying 'READ_MAX_LENGTH must be > 0 but got 0'.
To try you can, for example, download ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG00116/exome_alignment/HG00116.mapped.ILLUMINA.bwa.GBR.exome.20120522.bam
I'm using the b37 reference files from the bundle_gatk_2_8 and the bed file of SureSelect6 from Agilent .

Here the full stack trace and command line :

INFO 09:42:32,040 HelpFormatter - ------------------------------------------------------------------------------------
INFO 09:42:32,045 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.7-0-gcfedb67, Compiled 2016/12/12 11:21:18
INFO 09:42:32,045 HelpFormatter - Copyright (c) 2010-2016 The Broad Institute
INFO 09:42:32,045 HelpFormatter - For support and documentation go to https://software.broadinstitute.org/gatk
INFO 09:42:32,045 HelpFormatter - [Thu Apr 06 09:42:32 GMT 2017] Executing on Linux 2.6.18-275.12.1.el5.573g0000 amd64
INFO 09:42:32,045 HelpFormatter - Java HotSpot(TM) 64-Bit Server VM 1.8.0_40-b25
INFO 09:42:32,049 HelpFormatter - Program Args: -T MuTect2 -I:tumor /data/misc/mutect2/pon/1000g_bam/HG00116.mapped.ILLUMINA.bwa.GBR.exome.20120522.bam --db
snp /data/highlander/reference/bundle_gatk_2_8/b37/dbsnp_138.b37.vcf --artifact_detection_mode -L /data/highlander/reference/bundle_gatk_2_8/b37/capture.sureselect6.bed -R /data/highlander/reference/bundle_gatk_2_8/b37/human_g1k_v37.fasta -o /data/misc/mutect2/pon/1000g_vcf_normal/HG00116.vcf.gz
INFO 09:42:32,062 HelpFormatter - Executing as [email protected] on Linux 2.6.18-275.12.1.el5.573g0000 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_40-b25.
INFO 09:42:32,062 HelpFormatter - Date/Time: 2017/04/06 09:42:32
INFO 09:42:32,062 HelpFormatter - ------------------------------------------------------------------------------------
INFO 09:42:32,063 HelpFormatter - ------------------------------------------------------------------------------------

...

ERROR --
ERROR stack trace

java.lang.IllegalArgumentException: READ_MAX_LENGTH must be > 0 but got 0
at org.broadinstitute.gatk.utils.pairhmm.PairHMM.initialize(PairHMM.java:126)
at org.broadinstitute.gatk.utils.pairhmm.N2MemoryPairHMM.initialize(N2MemoryPairHMM.java:60)
at org.broadinstitute.gatk.utils.pairhmm.LoglessPairHMM.initialize(LoglessPairHMM.java:66)
at org.broadinstitute.gatk.utils.pairhmm.PairHMM.initialize(PairHMM.java:159)
at org.broadinstitute.gatk.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.initializePairHMM(PairHMMLikelihoodCalculationEngine.java:267)
at org.broadinstitute.gatk.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngine.java:282)
at org.broadinstitute.gatk.tools.walkers.cancer.m2.MuTect2.map(MuTect2.java:644)
at org.broadinstitute.gatk.tools.walkers.cancer.m2.MuTect2.map(MuTect2.java:171)
at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions$TraverseActiveRegionMap.apply(TraverseActiveRegions.java:709)
at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions$TraverseActiveRegionMap.apply(TraverseActiveRegions.java:705)
at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:274)
at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions.traverse(TraverseActiveRegions.java:274)
at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions.traverse(TraverseActiveRegions.java:78)
at org.broadinstitute.gatk.engine.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:98)
at org.broadinstitute.gatk.engine.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:316)
at org.broadinstitute.gatk.engine.CommandLineExecutable.execute(CommandLineExecutable.java:123)
at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)
at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
at org.broadinstitute.gatk.engine.CommandLineGATK.main(CommandLineGATK.java:108)

ERROR ------------------------------------------------------------------------------------------
ERROR A GATK RUNTIME ERROR has occurred (version 3.7-0-gcfedb67):
ERROR
ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
ERROR If not, please post the error message, with stack trace, to the GATK forum.
ERROR Visit our website and forum for extensive documentation and answers to
ERROR commonly asked questions https://software.broadinstitute.org/gatk
ERROR
ERROR MESSAGE: READ_MAX_LENGTH must be > 0 but got 0
ERROR ------------------------------------------------------------------------------------------

Am I doing something wrong or is it some kind of bug :-) ?

Thank you in advance for your help

Raphael

Answers

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @RaphaelH
    Hi Raphael,

    Can you try validating the BAM file with ValidateSamFile? Also, please post an example record from the BAM file here.

    Thanks,
    Sheila

  • RaphaelHRaphaelH BrusselsMember

    Hi Sheila,

    ValidateSamFile yields errors:
    ERROR: Record 217703738, Read name SRR099965.100486269, MAPQ should be 0 for unmapped read. ERROR: Record 217703740, Read name SRR099965.115232457, MAPQ should be 0 for unmapped read. ERROR: Record 217703742, Read name SRR099965.43304339, MAPQ should be 0 for unmapped read. ... ERROR: Record 217931432, Read name SRR099965.49982962, MAPQ should be 0 for unmapped read. ERROR: Record 217931456, Read name SRR099965.58007572, MAPQ should be 0 for unmapped read. Maximum output of [100] errors reached.
    The reads for the first error:

    samtools view HG00116.mapped.ILLUMINA.bwa.GBR.exome.20120522.bam | grep SRR099965.100486269 SRR099965.100486269 125 Y 59373560 37 5S7M1I63M = 59373560 0 AGACATCACGATGGATCACAGGTCTATCACCCTATTAACCACTCACGGGAGCTCTCCATGCATTTGGTATTTTCGT ######GC>HDFIGEFJDHHHGEIEDEICJJHABGCEBGIBJDKDAIIIHIJDJEJKDFIKDGGEHGCBFFFC>DC X0:i:1 X1:i:0 XC:i:71 MD:Z:0G2A0T65 RG:Z:SRR099965 AM:i:0 NM:i:4 SM:i:37 XN:i:7 XT:A:U SRR099965.100486269 189 Y 59373560 0 * = 59373560 0 GACCTGTGATCCATCGTGATGTCTTATTTAAGGGGAACGTGTGGGCTATTTAGGCTTTATGACCCTGAAGTAGGAA #?CFBECHCFEFBC>[email protected]> RG:Z:SRR099965

    I tried to redownload the BAM file from 1000g, checked md5sum, but I get always the same results.

    Thanks

    Raphael

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @RaphaelH
    Hi Raphael,

    You can use CleanSam to get rid of those errors.

    -Sheila

  • RaphaelHRaphaelH BrusselsMember

    Hi Sheila,

    I used CleanSam on all 1000G exome bam files, and relaunched ValidateSamFile which now says that everything is OK.

    [Thu May 04 14:19:10 CEST 2017] net.sf.picard.sam.ValidateSamFile INPUT=/data/misc/mutect2/pon/1000g_bam_cleaned/HG00116.bam MODE=VERBOSE MAX_OUTPUT=100 IGNORE_WARNINGS=false VALIDATE_INDEX=true IS_BISULFITE_SEQUENCED=false MAX_OPEN_TEMP_FILES=8000 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false
    Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /opt/picard-tools-1.107/libIntelDeflater.so which might have disabled stack guard. The VM will try to fix the stack guard now.
    It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'.
    [Thu May 04 14:19:10 CEST 2017] Executing as [email protected] on Linux 2.6.18-275.12.1.el5.573g0000 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_40-b25; Picard version: 1.107(1667) IntelDeflater
    INFO 2017-05-04 14:20:27 SamFileValidator Validated Read 10,000,000 records. Elapsed time: 00:01:16s. Time for last 10,000,000: 76s. Last read position: 1:246,498,968
    INFO 2017-05-04 14:21:44 SamFileValidator Validated Read 20,000,000 records. Elapsed time: 00:02:33s. Time for last 10,000,000: 76s. Last read position: 3:99,933,603
    INFO 2017-05-04 14:23:00 SamFileValidator Validated Read 30,000,000 records. Elapsed time: 00:03:49s. Time for last 10,000,000: 76s. Last read position: 5:139,906,749
    INFO 2017-05-04 14:24:16 SamFileValidator Validated Read 40,000,000 records. Elapsed time: 00:05:05s. Time for last 10,000,000: 76s. Last read position: 7:134,620,498
    INFO 2017-05-04 14:25:33 SamFileValidator Validated Read 50,000,000 records. Elapsed time: 00:06:22s. Time for last 10,000,000: 76s. Last read position: 10:76,782,051
    INFO 2017-05-04 14:26:49 SamFileValidator Validated Read 60,000,000 records. Elapsed time: 00:07:37s. Time for last 10,000,000: 75s. Last read position: 12:99,059,413
    INFO 2017-05-04 14:28:05 SamFileValidator Validated Read 70,000,000 records. Elapsed time: 00:08:54s. Time for last 10,000,000: 76s. Last read position: 16:3,433,208
    INFO 2017-05-04 14:29:21 SamFileValidator Validated Read 80,000,000 records. Elapsed time: 00:10:10s. Time for last 10,000,000: 76s. Last read position: 19:11,917,300
    INFO 2017-05-04 14:30:40 SamFileValidator Validated Read 90,000,000 records. Elapsed time: 00:11:29s. Time for last 10,000,000: 79s. Last read position: Y:17,466,931
    No errors found

    Unfortunately, I still get the same error with GATK:

    INFO 04:49:12,808 HelpFormatter - ------------------------------------------------------------------------------------
    INFO 04:49:12,811 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.7-0-gcfedb67, Compiled 2016/12/12 11:21:18
    INFO 04:49:12,811 HelpFormatter - Copyright (c) 2010-2016 The Broad Institute
    INFO 04:49:12,811 HelpFormatter - For support and documentation go to https://software.broadinstitute.org/gatk
    INFO 04:49:12,811 HelpFormatter - [Thu May 04 04:49:12 GMT 2017] Executing on Linux 2.6.18-275.12.1.el5.573g0000 amd64
    INFO 04:49:12,811 HelpFormatter - Java HotSpot(TM) 64-Bit Server VM 1.8.0_40-b25
    INFO 04:49:12,815 HelpFormatter - Program Args: -T MuTect2 -I:tumor /data/misc/mutect2/pon/1000g_bam_cleaned/HG00116.bam --dbsnp /data/highlander/reference/
    bundle_gatk_2_8/b37/dbsnp_138.b37.vcf --artifact_detection_mode -L /data/highlander/reference/bundle_gatk_2_8/b37/capture.sureselect6.bed -R /data/highlander
    /reference/bundle_gatk_2_8/b37/human_g1k_v37.fasta -o /data/misc/mutect2/pon/1000g_vcf_normal/HG00116.vcf.gz
    INFO 04:49:12,820 HelpFormatter - Executing as [email protected] on Linux 2.6.18-275.12.1.el5.573g0000 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_40-b25.
    INFO 04:49:12,820 HelpFormatter - Date/Time: 2017/05/04 04:49:12
    INFO 04:49:12,821 HelpFormatter - ------------------------------------------------------------------------------------
    INFO 04:49:12,821 HelpFormatter - ------------------------------------------------------------------------------------
    INFO 04:49:12,837 GenomeAnalysisEngine - Strictness is SILENT
    INFO 04:49:12,933 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 1000
    INFO 04:49:12,940 SAMDataSource$SAMReaders - Initializing SAMRecords in serial
    INFO 04:49:13,022 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.08
    INFO 04:49:16,986 IntervalUtils - Processing 100696230 bp from intervals
    INFO 04:49:17,094 GenomeAnalysisEngine - Preparing for traversal over 1 BAM files
    INFO 04:49:17,403 GenomeAnalysisEngine - Done preparing for traversal
    INFO 04:49:17,404 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING]
    INFO 04:49:17,404 ProgressMeter - | processed | time | per 1M | | total | remaining
    INFO 04:49:17,404 ProgressMeter - Location | active regions | elapsed | active regions | completed | runtime | runtime
    INFO 04:49:17,488 MuTect2 - Using global mismapping rate of 45 => -4.5 in log10 likelihood units
    WARN 04:49:18,392 PairHMMLikelihoodCalculationEngine$1 - Failed to load native library for VectorLoglessPairHMM - using Java implementation of LOGLESS_CACHI
    NG
    INFO 04:49:47,457 ProgressMeter - 1:1198932 5712089.0 30.0 s 5.0 s 0.1% 8.3 h 8.3 h
    INFO 04:50:17,468 ProgressMeter - 1:1342613 1.1725505E7 60.0 s 5.0 s 0.1% 11.3 h 11.2 h

    ...

    INFO 10:41:24,533 ProgressMeter - 2:165552411 1.05889674203E11 5.9 h 0.0 s 14.5% 40.3 h 34.5 h
    INFO 10:42:24,552 ProgressMeter - 2:166772053 1.0635087589E11 5.9 h 0.0 s 14.6% 40.3 h 34.4 h

    ERROR --
    ERROR stack trace

    java.lang.IllegalArgumentException: READ_MAX_LENGTH must be > 0 but got 0
    at org.broadinstitute.gatk.utils.pairhmm.PairHMM.initialize(PairHMM.java:126)
    at org.broadinstitute.gatk.utils.pairhmm.N2MemoryPairHMM.initialize(N2MemoryPairHMM.java:60)
    at org.broadinstitute.gatk.utils.pairhmm.LoglessPairHMM.initialize(LoglessPairHMM.java:66)
    at org.broadinstitute.gatk.utils.pairhmm.PairHMM.initialize(PairHMM.java:159)
    at org.broadinstitute.gatk.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.initializePairHMM(PairHMMLikelihoodCalculationEngine.java
    :267)
    at org.broadinstitute.gatk.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngine
    .java:282)
    at org.broadinstitute.gatk.tools.walkers.cancer.m2.MuTect2.map(MuTect2.java:644)
    at org.broadinstitute.gatk.tools.walkers.cancer.m2.MuTect2.map(MuTect2.java:171)
    at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions$TraverseActiveRegionMap.apply(TraverseActiveRegions.java:709)
    at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions$TraverseActiveRegionMap.apply(TraverseActiveRegions.java:705)
    at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:274)
    at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
    at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions.traverse(TraverseActiveRegions.java:274)
    at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions.traverse(TraverseActiveRegions.java:78)
    at org.broadinstitute.gatk.engine.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:98)
    at org.broadinstitute.gatk.engine.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:316)
    at org.broadinstitute.gatk.engine.CommandLineExecutable.execute(CommandLineExecutable.java:123)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
    at org.broadinstitute.gatk.engine.CommandLineGATK.main(CommandLineGATK.java:108)

    ERROR ------------------------------------------------------------------------------------------
    ERROR A GATK RUNTIME ERROR has occurred (version 3.7-0-gcfedb67):
    ERROR
    ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
    ERROR If not, please post the error message, with stack trace, to the GATK forum.
    ERROR Visit our website and forum for extensive documentation and answers to
    ERROR commonly asked questions https://software.broadinstitute.org/gatk
    ERROR
    ERROR MESSAGE: READ_MAX_LENGTH must be > 0 but got 0
    ERROR ------------------------------------------------------------------------------------------

    If I check the bam file for one of the reads which yield an error before with ValidateSam I get it but some flags have been changed

    samtools view /data/misc/mutect2/pon/1000g_bam_cleaned/HG00116.bam | grep SRR099965.100486269
    SRR099965.100486269 125 Y 59373560 0 5S7M1I63M = 59373560 0 AGACATCACGATGGATCACAGGTCTATCACCCTATTAACCACTCACGGGAGCTCTCCATGCATTTGGTATTTTCGT ######GC>HDFIGEFJDHHHGEIEDEICJJHABGCEBGIBJDKDAIIIHIJDJEJKDFIKDGGEHGCBFFFC>DC X0:i:1 X1:i:0 XC:i:71 MD:Z:0G2A0T65 RG:Z:SRR099965 AM:i:0 NM:i:4 SM:i:37 XN:i:7 XT:A:U
    SRR099965.100486269 189 Y 59373560 0 * = 59373560 0 GACCTGTGATCCATCGTGATGTCTTATTTAAGGGGAACGTGTGGGCTATTTAGGCTTTATGACCCTGAAGTAGGAA #?CFBECHCFEFBC>[email protected]> RG:Z:SRR099965

    Have you any idea of what could still cause the error in GATK ?

    Raphael

    Issue · Github
    by Sheila

    Issue Number
    2033
    State
    closed
    Last Updated
    Assignee
    Array
    Milestone
    Array
    Closed By
    chandrans
  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @RaphaelH
    Hi Raphael,

    I will have to check with the team. But, in the meantime, can you try to narrow down the BAM file to a small section to see where the error is occuring? For example, try running on each chromosome separately to see if at least some chromosomes are valid.

    Thanks,
    Sheila

  • RaphaelHRaphaelH BrusselsMember

    Hi Sheila,

    I ran each chromosome separately for the HG00116 sample and I get the error for every chromosome except 18 and 21 ... so it seems to occur nearly everywhere.
    I downloaded 80 exome samples from 1000G (CEU, GBR, IBS and TSI populations), and I get the problem with more than 70% of those bam files.

    Raphael

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @RaphaelH
    Hi Raphael,

    This may be a bug in MuTect2. The next version is coming out very soon, so I am thinking you may just have to wait and see if the issue is fixed in that version. You can also try out the alpha version in GATK4 here.

    -Sheila

  • RaphaelHRaphaelH BrusselsMember

    Hi Sheila,

    OK, I'll keep you informed if the problem solves itself in the next version and/or with GATK 4.

    Thanks again for you support !

    Raphael

  • SalvatoreSalvatore ItalyMember

    Hi
    Is the bug fixed in GATK4?
    Thanks
    Salvatore

    Issue · Github
    by shlee

    Issue Number
    2306
    State
    closed
    Last Updated
    Assignee
    Array
    Milestone
    Array
    Closed By
    chandrans
  • mark32309mark32309 New HavenMember

    Hi, I am also having this error using GATK 3.7-0-gcfedb67. I am trying to generate a PoN for WGS data. I will give GATK4.0 a try, but please let us know if there are any additional suggestions. Thanks for your help,

  • shleeshlee ✭✭✭✭✭ CambridgeMember, Broadie ✭✭✭✭✭

    Hi @mark32309,

    Yes, please do let us know if you still get the error with GATK4.

  • shleeshlee ✭✭✭✭✭ CambridgeMember, Broadie ✭✭✭✭✭

    @Salvatore, Sheila is away at a workshop and can follow up with you when she is back next week.

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @Salvatore
    Hi,

    I have not run into this issue in GATK4. Please test it out and let us know if BAM files that pass ValidateSamFile throw the error in GATK4.

    Thanks,
    Sheila

  • I have similar problem, but it's MuTect2 in GATK v3.8
    Program Args:
    GATK.jar
    java -Xmx4G -jar /home/2.Package/GATK3.8/GenomeAnalysisTK.jar
    -T MuTect2
    -R Hg19All_chr.fa
    -I:tumor NGS17I257-NGS-MMH-3-1-5_S24.bam
    -L chr21
    -nct 20
    -o NGS17I257-NGS-MMH-3-1-5_S24.vcf
    -stand_call_conf 20
    --dontUseSoftClippedBases

    ERROR ------------------------------------------------------------------------------------------
    ERROR A GATK RUNTIME ERROR has occurred (version 3.8-0-ge9d806836):
    ERROR
    ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
    ERROR If not, please post the error message, with stack trace, to the GATK forum.
    ERROR Visit our website and forum for extensive documentation and answers to
    ERROR commonly asked questions https://software.broadinstitute.org/gatk
    ERROR
    ERROR MESSAGE: READ_MAX_LENGTH must be > 0 but got 0
    ERROR ------------------------------------------------------------------------------------------

    However this problem doesn't show in the gatk-4.beta.5
    Program Args:
    gatk-launch \
    --javaOptions "-Xmx4g" Mutect2 \
    -R $ref_fa \
    -I $file \
    -tumor $TumorName \
    -L $chr \
    -threads $core \
    -O $Out \
    -stand_call_conf $Q \
    --dontUseSoftClippedBases >./log/Somatic${chr}-$i.log 2>&1

    My test run of Mutect2 is still running now, I'll feedback when it's done

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @TWIsaac
    Hi,

    Thanks for letting us know.

    -Sheila

  • valerijaarhvalerijaarh Member
    edited October 2017

    Hello,
    could you tell please, what is in result? I have the same problem with mutect2 (version gatk 3.8) and for me it is really important to run it. What shall I do?

    ERROR stack trace

    java.lang.IllegalArgumentException: READ_MAX_LENGTH must be > 0 but got 0
    at org.broadinstitute.gatk.utils.pairhmm.PairHMM.initialize(PairHMM.java:105)
    at org.broadinstitute.gatk.utils.pairhmm.N2MemoryPairHMM.initialize(N2MemoryPairHMM.java:60)
    at org.broadinstitute.gatk.utils.pairhmm.LoglessPairHMM.initialize(LoglessPairHMM.java:66)
    at org.broadinstitute.gatk.utils.pairhmm.PairHMM.initialize(PairHMM.java:138)
    at org.broadinstitute.gatk.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.initializePairHMM(PairHMMLikelihoodCalculationEngine.java:276)
    at org.broadinstitute.gatk.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngine.java:291)
    at org.broadinstitute.gatk.tools.walkers.cancer.m2.MuTect2.map(MuTect2.java:648)
    at org.broadinstitute.gatk.tools.walkers.cancer.m2.MuTect2.map(MuTect2.java:175)
    at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions$TraverseActiveRegionMap.apply(TraverseActiveRegions.java:709)
    at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions$TraverseActiveRegionMap.apply(TraverseActiveRegions.java:705)
    at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:274)
    at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
    at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions.traverse(TraverseActiveRegions.java:274)
    at org.broadinstitute.gatk.engine.traversals.TraverseActiveRegions.traverse(TraverseActiveRegions.java:78)
    at org.broadinstitute.gatk.engine.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:98)
    at org.broadinstitute.gatk.engine.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:323)
    at org.broadinstitute.gatk.engine.CommandLineExecutable.execute(CommandLineExecutable.java:123)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)
    at org.broadinstitute.gatk.engine.CommandLineGATK.main(CommandLineGATK.java:108)

    ERROR ------------------------------------------------------------------------------------------
    ERROR A GATK RUNTIME ERROR has occurred (version 3.8-0-ge9d806836):
    ERROR
    ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
    ERROR If not, please post the error message, with stack trace, to the GATK forum.
    ERROR Visit our website and forum for extensive documentation and answers to
    ERROR commonly asked questions https://software.broadinstitute.org/gatk
    ERROR
    ERROR MESSAGE: READ_MAX_LENGTH must be > 0 but got 0
    ERROR ------------------------------------------------------------------------------------------
  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @valerijaarh
    Hi,

    Have you validated your BAM file? Can you confirm this occurs with GATK4 beta 6?

    Thanks,
    Sheila

  • JulsJuls ✭✭ Member ✭✭
    edited November 2017

    Hi,
    I also ran into this issue with 3.8. I also need it quite urgently.
    I ran the HC on the same bam file, which ran through...
    Is it fixed with version 4?
    Thank you!

  • JulsJuls ✭✭ Member ✭✭
    edited November 2017

    On that note, I have run the recalibration on 3.8 - is it ok to use the Mutect2 4 on top of that?
    If it is indeed a bug, it would be great if it could be fixed with 3.8 as well...
    Thanks!

  • JulsJuls ✭✭ Member ✭✭

    I have now run cleanSam on the bam file before feeding it into Mutect2 3.8 and it still threw the error.

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @Juls
    Hi,

    What happens if you run ValidateSamFile on your BAM file?

    Can you check GATK4 beta 6? Another user says the issue does not occur in GATK4.

    -Sheila

  • JulsJuls ✭✭ Member ✭✭

    I haven't run ValidateSamFile on the BAM file yet - just cleanSam and the error still occurred with 3.8. It's currently running with GATK4 - it's not yet finished but it ran over the problematic site - so I thinks it's fine. However, I have run everything else on 3.8.

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @Juls
    Hi,

    I think that should be fine, as we are recommending Mutect2 in GATK4 over MuTect2 in GATK3. The pre-processing steps should not have changed much from GATK3 to GATK4 (at least not in the algorithms), so you should be okay.

    -Sheila

  • JulsJuls ✭✭ Member ✭✭
  • JulsJuls ✭✭ Member ✭✭

    I got this error now also with GATK 4 ... I am using GATK/4.0b//gatk-package-4.beta.5-local.jar

    02:10:50.070 INFO PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 50651.674885637
    02:10:50.070 INFO Mutect2 - Shutting down engine
    [February 4, 2018 2:10:50 AM CET] org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2 done. Elapsed time: 2,171.97 minutes.
    Runtime.totalMemory()=23542628352
    java.lang.IllegalArgumentException: readMaxLength must be > 0 but got 0

    Is there a bug fix yet?

  • dcampodcampo Los AngelesMember

    @Sheila said:
    @Juls
    Hi,

    I think that should be fine, as we are recommending Mutect2 in GATK4 over MuTect2 in GATK3. The pre-processing steps should not have changed much from GATK3 to GATK4 (at least not in the algorithms), so you should be okay.

    -Sheila

    Hi Sheila,

    Following up on this, is it OK to use a PoN created with MuTect2-GATK3 for somatic variant calling with Mutect2-GATK4?
    Thing is, my entire pipeline for somatic calling with RNAseq is based on STAR-GATK3, but when running MuTect2 I got the error mentioned above (READ_MAX_LENGTH must be > 0 but got 0) in about half of the bam files. I tried then running just the last step (i.e. Mutect2) with GATK4, and it worked. But I used the PoN I had from GATK3, so I am wondering if that's OK, or I need to re-do the PoN with GATK4.

    Thanks!

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @Juls
    Hi,

    Yes, there should be a fix in the very latest version of GATK4 (4.0.1.2).

    -Sheila

  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin
    edited February 2018

    @dcampo
    Hi,

    You should be fine using a PoN created with GATK3, as I don't think much has changed has changed in the actual algorithms (just the tools/workflow).

    -Sheila

    EDIT: Soo Hee pointed out that if you can spare the compute, it would be best to use GATK4 to create the PoN to keep consistency throughout your analysis. I only said you can use the GATK3 PoN since you already had it :smile:

    Post edited by Sheila on
  • JulsJuls ✭✭ Member ✭✭

    @Sheila
    I just got the error again with the current version (gatk-4.0.5.1):

    01:33:32.792 INFO  PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 116946.11351382501
    01:33:32.793 INFO  SmithWatermanAligner - Total compute time in java Smith-Waterman : 43239.15 sec
    01:33:32.794 INFO  Mutect2 - Shutting down engine
    [June 28, 2018 1:33:32 AM CEST] org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2 done. Elapsed time: 3,502.69 minutes.
    Runtime.totalMemory()=20542128128
    java.lang.IllegalArgumentException: readMaxLength must be > 0 but got 0
    
  • SheilaSheila admin Broad InstituteMember, Broadie, Moderator admin

    @Juls
    Hi,

    What happens when you run ValidateSamFile?

    -Sheila

  • manolismanolis ✭✭ Member ✭✭
    edited October 2018

    Hi,

    I'm using GATK 4.0.11.0 and I was trying to run Mutect2 locally.

    /home/manolis/bin/gatk-4.0.11.0/gatk --java-options "-Xmx5g" Mutect2 \
    -R ${hg38} \
    -I "${input}.bam" \
    -O "${output}.vcf.gz" \
    -tumor ${name} \
    --germline-resource af-only-gnomad.hg38.vcf.gz \
    -pon ${PON} \   #custom pon file
    -L ${hg38EXONS} \
    --bam-output "${output}.bam"
    

    After some time stop with the following error:

    Using GATK jar /home/manolis/bin/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar
    Running:
        java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx5g -jar /home/manolis/bin/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar Mutect2 -R /home/shared/resources/hgRef/hg38/Homo_sapiens_assembly38.fasta -I /home/manolis/GATK4/2.BQSR/ZMV_bqsr.bam -O ZMV_unmatched_M2.vcf.gz -tumor ZMV --germline-resource /home/shared/resources/gatk4hg38db/af-only-gnomad.hg38.vcf.gz -pon /home/manolis/GATK4/3.WES_Illumina/somSNV/Panel_of_Normals/storage/25xWES_20181018.vcf.gz -L /home/manolis/GATK4/DB/hg38_EXONSplus12_GENCODEv24_RefSeqCurated_noALTnoRANDOMnoCHRUNnoCHRM_sorted_merged.intervals --bam-output ZMV_unmatched_M2.bam
    15:21:58.168 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/manolis/bin/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
    15:21:59.916 INFO  Mutect2 - ------------------------------------------------------------
    15:21:59.917 INFO  Mutect2 - The Genome Analysis Toolkit (GATK) v4.0.11.0
    15:21:59.917 INFO  Mutect2 - For support and documentation go to https://software.broadinstitute.org/gatk/
    15:21:59.917 INFO  Mutect2 - Executing as [email protected] on Linux v4.4.0-131-generic amd64
    15:21:59.917 INFO  Mutect2 - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_121-b15
    15:21:59.918 INFO  Mutect2 - Start Date/Time: October 24, 2018 3:21:58 PM CEST
    15:21:59.918 INFO  Mutect2 - ------------------------------------------------------------
    15:21:59.918 INFO  Mutect2 - ------------------------------------------------------------
    15:21:59.919 INFO  Mutect2 - HTSJDK Version: 2.16.1
    15:21:59.919 INFO  Mutect2 - Picard Version: 2.18.13
    15:21:59.919 INFO  Mutect2 - HTSJDK Defaults.COMPRESSION_LEVEL : 2
    15:21:59.919 INFO  Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    15:21:59.919 INFO  Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    15:21:59.919 INFO  Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    15:21:59.919 INFO  Mutect2 - Deflater: IntelDeflater
    15:21:59.919 INFO  Mutect2 - Inflater: IntelInflater
    15:21:59.920 INFO  Mutect2 - GCS max retries/reopens: 20
    15:21:59.920 INFO  Mutect2 - Requester pays: disabled
    15:21:59.920 INFO  Mutect2 - Initializing engine
    15:22:00.723 INFO  FeatureManager - Using codec VCFCodec to read file file:///home/manolis/GATK4/3.WES_Illumina/somSNV/Panel_of_Normals/storage/25xWES_20181018.vcf.gz
    15:22:00.930 INFO  FeatureManager - Using codec VCFCodec to read file file:///home/shared/resources/gatk4hg38db/af-only-gnomad.hg38.vcf.gz
    15:22:01.778 INFO  IntervalArgumentCollection - Processing 129706928 bp from intervals
    15:22:02.056 INFO  Mutect2 - Done initializing engine
    15:22:02.165 INFO  NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/home/manolis/bin/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar!/com/intel/gkl/native/libgkl_utils.so
    15:22:02.191 INFO  PairHMM - OpenMP multi-threaded AVX-accelerated native PairHMM implementation is not supported
    15:22:02.191 WARN  PairHMM - ***WARNING: Machine does not have the AVX instruction set support needed for the accelerated AVX PairHmm. Falling back to the MUCH slower LOGLESS_CACHING implementation!
    15:22:02.472 INFO  ProgressMeter - Starting traversal
    15:22:02.473 INFO  ProgressMeter -        Current Locus  Elapsed Minutes     Regions Processed   Regions/Minute
    15:22:12.556 INFO  ProgressMeter -         chr1:1071913              0.2                   600           3570.4
    15:22:25.629 INFO  ProgressMeter -         chr1:1926710              0.4                  1560           4042.3
    ...
    ...
    15:33:02.748 INFO  ProgressMeter -       chr2:227699324             11.0                102370           9302.5
    15:33:12.755 INFO  ProgressMeter -       chr2:237429208             11.2                104560           9359.7
    15:33:16.657 INFO  PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 265.264776929
    15:33:16.657 INFO  SmithWatermanAligner - Total compute time in java Smith-Waterman : 36.63 sec
    15:33:17.622 INFO  Mutect2 - Shutting down engine
    [October 24, 2018 3:33:17 PM CEST] org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2 done. Elapsed time: 11.32 minutes.
    Runtime.totalMemory()=1959788544
    java.lang.IllegalArgumentException: readMaxLength must be > 0 but got 0
        at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:730)
        at org.broadinstitute.hellbender.utils.pairhmm.PairHMM.initialize(PairHMM.java:152)
        at org.broadinstitute.hellbender.utils.pairhmm.N2MemoryPairHMM.initialize(N2MemoryPairHMM.java:28)
        at org.broadinstitute.hellbender.utils.pairhmm.LoglessPairHMM.initialize(LoglessPairHMM.java:7)
        at org.broadinstitute.hellbender.utils.pairhmm.PairHMM.initialize(PairHMM.java:177)
        at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.initializePairHMM(PairHMMLikelihoodCalculationEngine.java:242)
        at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.PairHMMLikelihoodCalculationEngine.computeReadLikelihoods(PairHMMLikelihoodCalculationEngine.java:177)
        at org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2Engine.callRegion(Mutect2Engine.java:217)
        at org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2.apply(Mutect2.java:230)
        at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.processReadShard(AssemblyRegionWalker.java:291)
        at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.traverse(AssemblyRegionWalker.java:267)
        at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:966)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
        at org.broadinstitute.hellbender.Main.main(Main.java:289)
    

    I had the same error using also GATK 4.0.10.0

    I validated my input bam file and it is ok!

    Thanks

    Post edited by manolis on
  • shleeshlee ✭✭✭✭✭ CambridgeMember, Broadie ✭✭✭✭✭

    Hi @manolis,

    For followup, I've identified the pertinent code as:
    https://github.com/broadinstitute/gatk/blob/4.0.11.0/src/main/java/org/broadinstitute/hellbender/utils/pairhmm/PairHMM.java#L152

    Reading the javadoc in the code, it appears that reads with length = 0 are being passed into the pairHMM process. Can you think of why you may have reads with no length? If these are unmapped reads that haven't been annotated in a manner GATK expects, then CleanSam can clean them up for you.

    Cleans the provided SAM/BAM, soft-clipping beyond-end-of-reference alignments and setting MAPQ to 0 for unmapped reads

    GATK tools typically filter MAPQ0 reads so they do not reach the pairHMM step. If this doesn't solve the issue, then the next step I recommend is to try toggling settings for the ReadLengthReadFilter, which Mutect2 uses. For example, just to illustrate, you can disable it by adding --disable-read-filter ReadLengthReadFilter to your command. This particular read filter comes with two adjustable parameters. I see for v4.0.10.0:

    Valid only if "ReadLengthReadFilter" is specified:
    --max-read-length:Integer     Keep only reads with length at most equal to the specified value  Default value:
                                  2147483647. 
    
    --min-read-length:Integer     Keep only reads with length at least equal to the specified value  Default value: 30. 
    

    Given your command and error message:

    java.lang.IllegalArgumentException: readMaxLength must be > 0 but got 0
    at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:730)
    at org.broadinstitute.hellbender.utils.pairhmm.PairHMM.initialize(PairHMM.java:152)

    Let's test to see if in fact the default for --min-read-length is 30. Can you see if adding --min-read-length 30 to your command changes the outcome? If it does, please do let us know. If it does not, would you mind submitting a small snippet of your data that recapitulates this error? Let's try to solve the cause of this error. Instructions for bug reports are at https://software.broadinstitute.org/gatk/guide/article?id=1894. Thank you.

  • manolismanolis ✭✭ Member ✭✭

    Many thanks @shlee !

    I will let you know.

    Best

  • manolismanolis ✭✭ Member ✭✭

    Hi,

    the bam file that I'm using in this test was created according to the GATK4 pre-processing pipeline.

    I tested it with Mutect2 directly after BQSR (3.5Gb) as also after BQSR+CleanSam (2.5Gb).

    1) When I run the previously reported code (October 24) as a submitted Interactive Jobs I have always the above reported error (readMaxLength must be > 0 but got 0) and also this warning:

    15:22:02.191 WARN PairHMM - ***WARNING: Machine does not have the AVX instruction set support needed for the accelerated AVX PairHmm. Falling back to the MUCH slower LOGLESS_CACHING implementation!

    2) When I submit the code as batch jobs everything is ok, I do not have any error just the following warning:

    11:43:26.122 WARN IntelPairHmm - Flush-to-zero (FTZ) is enabled when running PairHMM.

    In the second case the vcf from the BQSR bam file is the same with the vcf from the BQSR+CleanSam bam file.

    I usually use Interactive Job just to test the codes, for data production I use batch jobs.

    Thanks

    Issue · Github
    by shlee

    Issue Number
    3165
    State
    closed
    Last Updated
    Assignee
    Array
    Closed By
    sooheelee
  • shleeshlee ✭✭✭✭✭ CambridgeMember, Broadie ✭✭✭✭✭

    Hi @manolis,

    BQSR (3.5Gb) as also after BQSR+CleanSam (2.5Gb).

    From the file sizes +/- CleanSam, it appears (i) your compression is different between the two processes and/or (ii) reads were dropped during CleanSam. Can you confirm the latter for us please?

    You've posted some observations about AVX in interactive mode and FTZ in batch mode. Do you run these on different systems? Also,is there anything you can think of about how you do batch processing that would subset the data differently?

  • manolismanolis ✭✭ Member ✭✭
    edited November 2018

    Hi @shlee ,

    here the code of "ApplyBQSR"

    Running:
        java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx10g -XX:ConcGCThreads=1 -jar /home/manolis/bin/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar ApplyBQSR -R /shared/resources/hgRef/hg38/Homo_sapiens_assembly38.fasta -I /home/manolis/GATK4/1B.pre-processing_Fastq_TO_FixedBAM/fixed_bam/ZMV_fixed.bam -O ZMV_bqsr.bam -bqsr ZMV_1st_recaldata.csv --static-quantized-quals 10 --static-quantized-quals 20 --static-quantized-quals 30 --add-output-sam-program-record --create-output-
    bam-md5 --use-original-qualities
    11:41:52.773 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/manolis/bin/gatk-4.0.11.0/gatk-package-4.0.11.0-local.jar
    !/com/intel/gkl/native/libgkl_compression.so
    

    and here of "CleanSam"

    21:40:37.638 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/share/apps/bio/picard-2.17.3/picard.jar!/com/intel/gkl/native/libgkl_compression.so
    [Thu Nov 01 21:40:37 CET 2018] CleanSam INPUT=ZMV_bqsr.bam OUTPUT=ZMV.txt    VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false
    

    The codes use two differente compression leves, the first 2 and the second one 5

    I do not know the message when the code drop reads...

    Thu Nov 01 21:40:37 CET 2018] Executing as [email protected] on Linux 4.4.0-128-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_121-b15; Deflater: Intel; Inflater: Intel; Picard version: 2.17.3-SNAPSHOT
    INFO 2018-11-01 21:40:51 CleanSam Processed 1,000,000 records. Elapsed time: 00:00:13s. Time for last 1,000,000: 13s. Last read position: chr1:27,105,800
    INFO 2018-11-01 21:41:02 CleanSam Processed 2,000,000 records. Elapsed time: 00:00:24s. Time for last 1,000,000: 11s. Last read position: chr1:78,660,654
    INFO 2018-11-01 21:41:13 CleanSam Processed 3,000,000 records. Elapsed time: 00:00:35s. Time for last 1,000,000: 10s. Last read position: chr1:154,268,734
    INFO 2018-11-01 21:41:24 CleanSam Processed 4,000,000 records. Elapsed time: 00:00:46s. Time for last 1,000,000: 10s. Last read position: chr1:204,464,508
    INFO 2018-11-01 21:41:34 CleanSam Processed 5,000,000 records. Elapsed time: 00:00:56s. Time for last 1,000,000: 10s. Last read position: chr2:24,299,902
    INFO 2018-11-01 21:41:45 CleanSam Processed 6,000,000 records. Elapsed time: 00:01:07s. Time for last 1,000,000: 11s. Last read position: chr2:95,117,876
    INFO 2018-11-01 21:41:58 CleanSam Processed 7,000,000 records. Elapsed time: 00:01:20s. Time for last 1,000,000: 12s. Last read position: chr2:174,134,739
    INFO 2018-11-01 21:42:08 CleanSam Processed 8,000,000 records. Elapsed time: 00:01:30s. Time for last 1,000,000: 10s. Last read position: chr2:239,401,919
    INFO 2018-11-01 21:42:19 CleanSam Processed 9,000,000 records. Elapsed time: 00:01:41s. Time for last 1,000,000: 10s. Last read position: chr3:49,700,232
    INFO 2018-11-01 21:42:30 CleanSam Processed 10,000,000 records. Elapsed time: 00:01:52s. Time for last 1,000,000: 10s. Last read position: chr3:129,481,677
    ...
    ...

    Unfortunately I do not have experience about server systems and the structure of our server :neutral:

    batch mode

    queuename qtype resv/used/tot. load_avg arch states

    [email protected] BP 0/30/62 34.84 lx26-amd64

    Interactive mode

    queuename qtype resv/used/tot. load_avg arch states

    [email protected] IP 0/0/16 0.02 lx26-amd64

  • shleeshlee ✭✭✭✭✭ CambridgeMember, Broadie ✭✭✭✭✭

    Hi @manolis,

    I will consult with a developer and get back to you.

  • shleeshlee ✭✭✭✭✭ CambridgeMember, Broadie ✭✭✭✭✭

    Hi @manolis,

    For the original error:

    java.lang.IllegalArgumentException: readMaxLength must be > 0 but got 0

    Our developers ask if you can check your data for reads without any bases. Thanks.

  • manolismanolis ✭✭ Member ✭✭
    edited November 2018

    /share/apps/bio/samtools/samtools view -f 4

    ZMV_bqsr.bam file

    all reads

    48791770

    check for unmapped reads

    63003

    /share/apps/bio/samtools/samtools view ZMV_bqsr.bam | awk '{print length($10)}' | sort -n | uniq -c

        160 20
        147 21
        179 22
        162 23
        188 24
        200 25
        214 26
        203 27
        238 28
        234 29
        242 30
        282 31
        226 32
        264 33
        334 34
        305 35
        329 36
        358 37
        457 38
        467 39
        544 40
        620 41
        730 42
       1247 43
        704 44
        975 45
        842 46
        840 47
        988 48
       1067 49
       1029 50
        989 51
       1196 52
       1446 53
       2347 54
       1962 55
       2008 56
       2047 57
       1980 58
       2193 59
       2161 60
       2335 61
       2333 62
       2171 63
       2824 64
       2541 65
       2831 66
       2126 67
       2903 68
       2336 69
       2322 70
       2314 71
       2342 72
       2232 73
       2204 74
       2226 75
       2335 76
       2386 77
       2306 78
       2479 79
       2497 80
       2488 81
       2588 82
       2545 83
       2733 84
       2681 85
       2965 86
       2868 87
       2992 88
       3229 89
       3499 90
       3674 91
       3852 92
       3951 93
       4297 94
       4476 95
       4943 96
       5218 97
       5760 98
       6332 99
       6928 100
       7428 101
       8644 102
       9102 103
      10182 104
      11401 105
      12992 106
      14039 107
      15546 108
      17557 109
      19455 110
      21273 111
      23640 112
      26141 113
      28536 114
      31911 115
      34819 116
      37776 117
      41934 118
      45542 119
      49306 120
      53396 121
      58477 122
      63304 123
      68110 124
      73817 125
      79180 126
      85580 127
      91015 128
      95800 129
     102122 130
     108643 131
     115202 132
     121327 133
     130093 134
     138198 135
     146776 136
     151483 137
     162201 138
     178355 139
     211780 140
     313610 141
     620321 142
    1693687 143
    4894290 144
    11588659 145
    26819954 146
    

    I don't know if you mean those information...

  • Hi,

    I got the same error "READ_MAX_LENGTH must be > 0 but got 0" when running Mutect2 (GATK 4.0.11.0) only on some processed bam files (most other bam files are okay). But when I changed to GATK version 4.0.10.1, all of them ran completely okay.
    So I think this could be version specific issue.

    Thanks!

  • SChaluvadiSChaluvadi admin Member, Broadie, Moderator admin

    Hi @Phuong_Nguyen_1

    When you ran Mutect2 with GATK 4.0.11.0, did it crash? Would you be able to provide a stack trace and the code snippet that you were using as well? It will help troubleshoot faster! Thank you!

  • @SChaluvadi said:
    Hi @Phuong_Nguyen_1

    When you ran Mutect2 with GATK 4.0.11.0, did it crash? Would you be able to provide a stack trace and the code snippet that you were using as well? It will help troubleshoot faster! Thank you!

    Hi,

    I am very sorry, that error still appeared in other version of GATK (4.0.8.1 and others newer), so it was not version specific issue, my mistake.

    I start to run some test to look for the reason, when I run the SAME version, SAME command (detail below) on different machines, 1 gave SUCCESS (log..before.txt) and the other gave error "READ_MAX_LENGTH must be > 0 but got 0" (log..after.txt). I could only be able to send you those 2 log files as attachments.

    Here is the run command:
    ${gatk} --java-options "${mem}" Mutect2 \
    -R ${genome} \
    -I ${dir}/${T}/${T}.processed.bam \
    -I ${dir}/${N}/${N}.processed.bam \
    -tumor ${T} \
    -normal ${N} \
    --germline-resource ${db} \
    --af-of-alleles-not-in-resource 0.0000025 \
    -O T-N_${v}${i}.vcf.gz \
    -bamout T-N_${v}${i}.bam &> log.T-N_${v}${i}.Mutect2

    The only different between 2 machines is AVX structure, therefore GATK give this warning:
    10:31:03.566 INFO PairHMM - OpenMP multi-threaded AVX-accelerated native PairHMM implementation is not supported
    10:31:03.566 WARN PairHMM - ***WARNING: Machine does not have the AVX instruction set support needed for the accelerated AVX PairHmm. Falling back to the MUCH slower LOGLESS_CACHING implementation!

    In short, I suppose that this was a complex error which was contributed by both the bam file itself and the platform structure running it. (Note that I only encounter this error on 1/40 samples in the same pipeline)

    I hope this can do some help.

    Best regards,
    Phuong

  • SChaluvadiSChaluvadi admin Member, Broadie, Moderator admin
    edited December 2018

    @Phuong_Nguyen_1
    The accelerated AVX PairHmm and LOGLESS_CACHING PairHMM are two different implementations, and it looks like there may be bug somewhere in the LOGLESS_CACHING implementation since only some samples fail which indicates that the bug is an edge case that most samples don't hit. A bug report has been filed and you can follow along with the ticket.

    Post edited by SChaluvadi on
  • JulsJuls ✭✭ Member ✭✭

    @Sheila
    @AdelaideR

    READ_MAX_LENGTH must be > 0 but got 0

    I was wondering if a fixed 3.8 Mutect2 version was ever released?
    Thanks!

  • SChaluvadiSChaluvadi admin Member, Broadie, Moderator admin

    @Juls
    Looks like based on the ticket linked above, there haven't been any posted updates on a bug release.

  • JulsJuls ✭✭ Member ✭✭

    @SChaluvadi
    Thanks for your quick reply. Do you think they will fix it in the (near) future?

  • tonytony ✭✭ Member ✭✭

    Hi all,

    @SChaluvadi, I want to back up @Juls request on this. I am using GATK 4.1.0.0 and I have hit this error MANY times either when creating panels or searching for somatic mutations with Mutect2 with both WES and WGS data.

    Basically I am just wondering whether or not there is some kind of high priority set on this point ?
    It makes more or less Mutect2 not really usable these times.

    Are there any backup solution ? (specific version to use for instance)

    Related GitHub issue is https://github.com/broadinstitute/gatk/issues/5543 but is not set as a high priority bug.

    Many thanks,
    Anthony

  • SChaluvadiSChaluvadi admin Member, Broadie, Moderator admin

    @tony @Juls
    I will ask the development team and get back to you both with an answer!

  • SChaluvadiSChaluvadi admin Member, Broadie, Moderator admin

    @tony @Juls
    We've just put a development teammate on the issue but I think we'll see some progress soon. Someone has also responded to the Github issue so you can follow along there as well!

  • tonytony ✭✭ Member ✭✭

    Many thanks @SChaluvadi !
    Let's hope it's not a hard fix.

  • JulsJuls ✭✭ Member ✭✭

    Many thanks @SChaluvadi!
    I also hope it's not that hard to fix.

  • suimyesuimye TokyoMember

    Hi everyone,
    This is only reporting from me.
    got the same error "READ_MAX_LENGTH must be > 0 but got 0" when running Mutect2 on docker (broadinstitute/gatk, version:4.0.8.1 and imageID: 123712a62f94). While most others were the success to finish calling, only a few processed bam files were failed. According to this discussion, I have tried to use both ValidateSamFile and CleanSam commands. But I could not resolve this problem. When I changed to GATK docker nightly version (imageID:6dc75ccab43b), Mutect2 jobs completely finished.

    The latest version of GATK (download 2019.4.3) may be fixed this issue.

Sign In or Register to comment.