Removed too many insertions, header is now negative!

bpowbpow Posts: 6Member

New gatk version... trying out ReduceReads again.

6 of 8 exomes I tried were processed by ReduceReads just fine, but two throw the exception Removed too many insertions, header is now negative! (at different genomic locations).

I did not find any mention of this error in the GATK forums, is this a known problem?

Command line: java -Xmx6g -jar GenomeAnalysisTK.jar -R human_g1k_v37.fasta -T ReduceReads -o test.rr.bam -I rr-too-many-insertions.bam

java -v: java version "1.6.0_27" Java(TM) SE Runtime Environment (build 1.6.0_27-b07) Java HotSpot(TM) 64-Bit Server VM (build 20.2-b06, mixed mode)

Run log:

INFO  16:03:26,898 HelpFormatter - -------------------------------------------------------------------------------- 
INFO  16:03:27,382 HelpFormatter - The Genome Analysis Toolkit (GATK) v2.3-0-g9593e74, Compiled 2012/12/17 16:58:19 
INFO  16:03:27,383 HelpFormatter - Copyright (c) 2010 The Broad Institute 
INFO  16:03:27,383 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk 
INFO  16:03:27,388 HelpFormatter - Program Args: -R human_g1k_v37.fasta -T ReduceReads -o test.rr.bam -I rr-too-many-insertions.bam 
INFO  16:03:27,388 HelpFormatter - Date/Time: 2012/12/18 16:03:26 
INFO  16:03:27,388 HelpFormatter - -------------------------------------------------------------------------------- 
INFO  16:03:27,388 HelpFormatter - -------------------------------------------------------------------------------- 
INFO  16:03:27,471 GenomeAnalysisEngine - Strictness is SILENT 
INFO  16:03:27,577 GenomeAnalysisEngine - Downsampling Settings: No downsampling 
INFO  16:03:27,585 SAMDataSource$SAMReaders - Initializing SAMRecords in serial 
INFO  16:03:27,620 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.03 
INFO  16:03:27,656 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING] 
INFO  16:03:27,657 ProgressMeter -        Location processed.reads  runtime per.1M.reads completed total.runtime remaining 
INFO  16:03:27,714 ReadShardBalancer$1 - Loading BAM index data for next contig 
INFO  16:03:27,717 ReadShardBalancer$1 - Done loading BAM index data for next contig 
INFO  16:03:27,739 ReadShardBalancer$1 - Loading BAM index data for next contig 
INFO  16:03:28,739 GATKRunReport - Uploaded run statistics report to AWS S3 
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR stack trace 
org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Removed too many insertions, header is now negative!
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.HeaderElement.removeInsertionToTheRight(HeaderElement.java:151)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.updateHeaderCounts(SlidingWindow.java:881)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.removeFromHeader(SlidingWindow.java:816)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.compressVariantRegion(SlidingWindow.java:604)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegion(SlidingWindow.java:623)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegions(SlidingWindow.java:643)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SingleSampleCompressor.closeVariantRegions(SingleSampleCompressor.java:83)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.MultiSampleCompressor.closeVariantRegionsInAllSamples(MultiSampleCompressor.java:94)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.MultiSampleCompressor.addAlignment(MultiSampleCompressor.java:76)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReadsStash.compress(ReduceReadsStash.java:67)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReads.reduce(ReduceReads.java:387)
    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReads.reduce(ReduceReads.java:87)
    at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano$TraverseReadsReduce.apply(TraverseReadsNano.java:226)
    at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano$TraverseReadsReduce.apply(TraverseReadsNano.java:215)
    at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:254)
    at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:219)
    at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano.traverse(TraverseReadsNano.java:91)
    at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano.traverse(TraverseReadsNano.java:55)
    at org.broadinstitute.sting.gatk.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:83)
    at org.broadinstitute.sting.gatk.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:281)
    at org.broadinstitute.sting.gatk.CommandLineExecutable.execute(CommandLineExecutable.java:113)
    at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:237)
    at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:147)
    at org.broadinstitute.sting.gatk.CommandLineGATK.main(CommandLineGATK.java:94)
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR A GATK RUNTIME ERROR has occurred (version 2.3-0-g9593e74):
##### ERROR
##### ERROR Please visit the wiki to see if this is a known problem
##### ERROR If not, please post the error, with stack trace, to the GATK forum
##### ERROR Visit our website and forum for extensive documentation and answers to 
##### ERROR commonly asked questions http://www.broadinstitute.org/gatk
##### ERROR
##### ERROR MESSAGE: Removed too many insertions, header is now negative!
##### ERROR ------------------------------------------------------------------------------------------

(there is no progress listed here because this log is from after I bisected to find a narrow region where the problem is occuring).

Best Answer

Answers

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 6,804Administrator, GATK Developer admin

    Hmm, that's a new one. Could you please upload the slice of BAM where you're encountering the error?

    Instructions here if needed: http://www.broadinstitute.org/gatk/guide/article?id=1894

    Geraldine Van der Auwera, PhD

  • bpowbpow Posts: 6Member

    I uploaded a set of reads that gives the error in the archive rr-too-many-insertions.tar.gz

    Thanks.

  • ebanksebanks Posts: 684GATK Developer mod

    Hi there,

    Thank you very much for reporting this bug and providing a test file - it was extremely helpful! I just put together a fix that will be available in the next release (2.4).

    Eric Banks, PhD -- Senior Group Leader, MPG Analysis, Broad Institute of Harvard and MIT

  • bpowbpow Posts: 6Member
    edited December 2012

    Thanks for looking into this so promptly (and for putting the fix in a patch release instead of waiting for the next point version).

    I've marked this as answered since I can now successfully run ReduceReads on my samples. Now I'm back to getting the "Somehow the requested coordinate is not covered by the read. Too many deletions?" error when I try the HaplotypeCaller. Since that's a separate issue, I'll address it its previous thread if I can't find anything wrong with my input files.

    Post edited by bpow on
  • croceacrocea Posts: 12Member

    I'm using GATK 2.5-2-gf57256b. still have this error. shall i upload the bam file?

    ...$ /u/home/eeskin/polyacti/bin/jdk/bin/java  -Xms4200m -Xmx5600m -XX:PermSize=1050m -XX:MaxPermSize=1400m -jar  gatk2Jar/GenomeAnalysisTK.jar  --reference_sequence  reference/3280_vervet_ref_6.0.3.fasta  -T ReduceReads  --unsafe --validation_strictness SILENT --read_filter BadCigar  -L Scaffold1101:1-576 -I  individual_alignment/2503_841_vs_3280_by_2.bam  --out  folderMap/2503_841_2008074_GA_vs_3280_Scaffold1101_1_576.reduceReads.bam  --downsample_to_coverage 60 --downsampling_type BY_SAMPLE
    
    WARNING: BAM index file /u/home/eeskin2/polyacti/NetworkData/scratch/ReduceReads/ReduceReadsAlnID2470_2509_Ref3280.2013.
    May.8T195100/individual_alignment/2503_841_vs_3280_by_2.bam.bai is older than BAM /u/home/eeskin2/polyacti/NetworkData/scratch/Red
    uceReads/ReduceReadsAlnID2470_2509_Ref3280.2013.May.8T195100/individual_alignment/2503_841_vs_3280_by_2.bam
    INFO  15:59:20,252 HelpFormatter - -------------------------------------------------------------------------------- 
    INFO  15:59:20,254 HelpFormatter - The Genome Analysis Toolkit (GATK) v2.5-2-gf57256b, Compiled 2013/05/01 09:27:02 
    INFO  15:59:20,255 HelpFormatter - Copyright (c) 2010 The Broad Institute 
    INFO  15:59:20,255 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk 
    INFO  15:59:20,258 HelpFormatter - Program Args: --reference_sequence reference/3280_vervet_ref_6.0.3.fasta -T ReduceReads --unsaf
    e --validation_strictness SILENT --read_filter BadCigar -L Scaffold1101:1-576 -I individual_alignment/2503_841_vs_3280_by_2.bam --
    out folderMap/2503_841_2008074_GA_vs_3280_Scaffold1101_1_576.reduceReads.bam --downsample_to_coverage 60 --downsampling_type BY_SA
    MPLE 
    INFO  15:59:20,259 HelpFormatter - Date/Time: 2013/05/13 15:59:20 
    INFO  15:59:20,259 HelpFormatter - -------------------------------------------------------------------------------- 
    INFO  15:59:20,259 HelpFormatter - -------------------------------------------------------------------------------- 
    INFO  15:59:21,155 GenomeAnalysisEngine - Strictness is SILENT 
    INFO  15:59:21,497 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 60 
    INFO  15:59:21,504 SAMDataSource$SAMReaders - Initializing SAMRecords in serial 
    INFO  15:59:21,556 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.05 
    INFO  15:59:21,570 IntervalUtils - Processing 576 bp from intervals 
    INFO  15:59:21,678 GenomeAnalysisEngine - Creating shard strategy for 1 BAM files 
    INFO  15:59:21,702 GenomeAnalysisEngine - Done creating shard strategy 
    INFO  15:59:21,702 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING] 
    INFO  15:59:21,703 ProgressMeter -        Location processed.reads  runtime per.1M.reads completed total.runtime remaining 
    INFO  15:59:21,837 ReadShardBalancer$1 - Loading BAM index data for next contig 
    INFO  15:59:21,838 ReadShardBalancer$1 - Done loading BAM index data for next contig 
    INFO  15:59:21,865 ReadShardBalancer$1 - Loading BAM index data for next contig 
    INFO  15:59:23,494 GATKRunReport - Uploaded run statistics report to AWS S3 
    ##### ERROR ------------------------------------------------------------------------------------------
    ##### ERROR stack trace 
    org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Removed too many insertions, header is now negative!
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.HeaderElement.removeInsertionToTheRight(HeaderElement.jav
    a:210)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.actuallyUpdateHeaderForRead(SlidingWindow.j
    ava:1242)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.updateHeaderCounts(SlidingWindow.java:1158)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.removeFromHeader(SlidingWindow.java:1135)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.compressVariantRegion(SlidingWindow.java:74
    2)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegion(SlidingWindow.java:835)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegions(SlidingWindow.java:880)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegions(SlidingWindow.java:850)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SingleSampleCompressor.closeVariantRegions(SingleSampleCo
    mpressor.java:149)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.MultiSampleCompressor.closeVariantRegionsInAllSamples(Mul
    tiSampleCompressor.java:158)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.MultiSampleCompressor.addAlignment(MultiSampleCompressor.
    java:127)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReadsStash.compress(ReduceReadsStash.java:116)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReads.reduce(ReduceReads.java:453)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReads.reduce(ReduceReads.java:112)
            at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano$TraverseReadsReduce.apply(TraverseReadsNano.java:253)
            at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano$TraverseReadsReduce.apply(TraverseReadsNano.java:242)
            at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:279)
            at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
            at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano.traverse(TraverseReadsNano.java:102)
            at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano.traverse(TraverseReadsNano.java:56)
            at org.broadinstitute.sting.gatk.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:109)
            at org.broadinstitute.sting.gatk.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:286)
            at org.broadinstitute.sting.gatk.CommandLineExecutable.execute(CommandLineExecutable.java:113)
            at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:245)
            at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:152)
            at org.broadinstitute.sting.gatk.CommandLineGATK.main(CommandLineGATK.java:91)
    ##### ERROR ------------------------------------------------------------------------------------------
    ##### ERROR A GATK RUNTIME ERROR has occurred (version 2.5-2-gf57256b):
    ##### ERROR
    ##### ERROR Please check the documentation guide to see if this is a known problem
    ##### ERROR If not, please post the error, with stack trace, to the GATK forum
    ##### ERROR Visit our website and forum for extensive documentation and answers to 
    ##### ERROR commonly asked questions http://www.broadinstitute.org/gatk
    ##### ERROR
    ##### ERROR MESSAGE: Removed too many insertions, header is now negative!
    ##### ERROR ------------------------------------------------------------------------------------------
    
  • CarneiroCarneiro Posts: 274Administrator, GATK Developer admin

    Hi Crocea, can you try the nightly build and see if it still happens ? we have pushed a recent fix on this issue that could resolve your problem.

  • croceacrocea Posts: 12Member

    ok. it's running and seems to be working. thanks

    @Carneiro said: Hi Crocea, can you try the nightly build and see if it still happens ? we have pushed a recent fix on this issue that could resolve your problem.

  • croceacrocea Posts: 12Member

    I have to retract what i said. this error is still there. with latest nightly. do you need the bam file?

    ##### ERROR ------------------------------------------------------------------------------------------
    ##### ERROR stack trace 
    org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Removed too many insertions, header is now negative!
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.HeaderElement.removeInsertionToTheRight(HeaderElement.java:210)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.actuallyUpdateHeaderForRead(SlidingWindow.java:1246)
            at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.updateHeaderCounts(SlidingWindow.java:1162)
    ##### ERROR ------------------------------------------------------------------------------------------
    ##### ERROR A GATK RUNTIME ERROR has occurred (version nightly-2013-05-17-g2c8b717):
    ##### ERROR
    ...
    ##### ERROR
    ##### ERROR MESSAGE: Removed too many insertions, header is now negative!
    ##### ERROR ------------------------------------------------------------------------------------------
    

    @crocea said: ok. it's running and seems to be working. thanks

    @Carneiro said: Hi Crocea, can you try the nightly build and see if it still happens ? we have pushed a recent fix on this issue that could resolve your problem.

  • ebanksebanks Posts: 684GATK Developer mod

    Yes, please. Please see the instructions on how to upload test files. Thanks

    Eric Banks, PhD -- Senior Group Leader, MPG Analysis, Broad Institute of Harvard and MIT

  • croceacrocea Posts: 12Member

    Hi eric,

    I uploaded the segment of the bam that caused the error. I followed this post , http://gatkforums.broadinstitute.org/discussion/1215/how-can-i-access-the-gsa-public-ftp-server#latest, and uploaded the file (132K) , "2317_645_vs_3280_by_2.Scaffold739.bam" to ftp://ftp.broadinstitute.org, the folder is "/ReduceReadsBug".

    thanks, yu

    @ebanks said: Yes, please. Please see the instructions on how to upload test files. Thanks

  • ebanksebanks Posts: 684GATK Developer mod

    Hi Yu,

    Unfortunately we can't work with the data you provided. The post containing instructions for uploading test files is located here: http://gatkforums.broadinstitute.org/discussion/1894/how-do-i-submit-a-detailed-bug-report Thanks

    Eric Banks, PhD -- Senior Group Leader, MPG Analysis, Broad Institute of Harvard and MIT

  • croceacrocea Posts: 12Member

    OK. now i uploaded it with commandline, stack trace, fasta file (non-standard species, ), it's at ReduceReadsBug/ReduceReadsBug.tar.gz on ftp://ftp.broadinstitute.org.

    thanks,

    @ebanks said: Hi Yu,

    Unfortunately we can't work with the data you provided. The post containing instructions for uploading test files is located here: http://gatkforums.broadinstitute.org/discussion/1894/how-do-i-submit-a-detailed-bug-report Thanks

  • croceacrocea Posts: 12Member

    it's 835Mb due to the fasta file, still uploading.

    @crocea said: OK. now i uploaded it with commandline, stack trace, fasta file (non-standard species, ), it's at ReduceReadsBug/ReduceReadsBug.tar.gz on ftp://ftp.broadinstitute.org.

    thanks,

    Thanks

  • ebanksebanks Posts: 684GATK Developer mod

    Thanks for the perfect test files! I was able to find the problem and hopefully a fix will be in the nightly builds starting tonight.

    Eric Banks, PhD -- Senior Group Leader, MPG Analysis, Broad Institute of Harvard and MIT

  • croceacrocea Posts: 12Member

    All my reduceReads workflows are firing at all cylinders right now... Thanks for the fix. Eric.

    @ebanks said: Thanks for the perfect test files! I was able to find the problem and hopefully a fix will be in the nightly builds starting tonight.

  • JulsJuls Graz, AustriaPosts: 4Member

    Same issue here - I am now downloading the nightly build

    ERROR ------------------------------------------------------------------------------------------
    ERROR stack trace

    org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Removed too many insertions, header is now negative at position 1436231

  • vplagnolvplagnol Posts: 51Member
    edited July 2013

    Just to add to this: I see the same issue for some BAM files, with version 2.6-4. As far as I can tell the exact same BAM files get reduced OK with 2.5-2. It looks like something happened between releases.

    Post edited by vplagnol on
  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 6,804Administrator, GATK Developer admin

    @Juls and @vplagnol, are you still seeing this bug with the latest nightly build?

    Geraldine Van der Auwera, PhD

  • umauma virginiaPosts: 10Member

    Hi, I am running the latest version of GATK 2.6 and i find the same issue:

    ERROR ------------------------------------------------------------------------------------------
    ERROR stack trace

    org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Removed too many insertions, header is now negative at position 27085688 at org.broadinstitute.sting.gatk.walkers.compression.reducereads.HeaderElement.removeInsertionToTheRight(HeaderElement.java:210) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.actuallyUpdateHeaderForRead(SlidingWindow.java:1246) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.updateHeaderCounts(SlidingWindow.java:1162) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.removeFromHeader(SlidingWindow.java:1139) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.compressVariantRegion(SlidingWindow.java:742) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegion(SlidingWindow.java:835) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegions(SlidingWindow.java:884) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.close(SlidingWindow.java:971) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SingleSampleCompressor.addAlignment(SingleSampleCompressor.java:113) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.MultiSampleCompressor.addAlignment(MultiSampleCompressor.java:123) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReadsStash.compress(ReduceReadsStash.java:116) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReads.reduce(ReduceReads.java:477) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReads.reduce(ReduceReads.java:113) at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano$TraverseReadsReduce.apply(TraverseReadsNano.java:251) at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano$TraverseReadsReduce.apply(TraverseReadsNano.java:240) at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:279) at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245) at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano.traverse(TraverseReadsNano.java:102) at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano.traverse(TraverseReadsNano.java:56) at org.broadinstitute.sting.gatk.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:108) at org.broadinstitute.sting.gatk.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:311) at org.broadinstitute.sting.gatk.CommandLineExecutable.execute(CommandLineExecutable.java:113) at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:245) at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:152) at org.broadinstitute.sting.gatk.CommandLineGATK.main(CommandLineGATK.java:91)

    ERROR ------------------------------------------------------------------------------------------
    ERROR A GATK RUNTIME ERROR has occurred (version 2.6-4-g3e5ff60):
    ERROR
    ERROR Please check the documentation guide to see if this is a known problem
    ERROR If not, please post the error, with stack trace, to the GATK forum
    ERROR Visit our website and forum for extensive documentation and answers to
    ERROR commonly asked questions http://www.broadinstitute.org/gatk
    ERROR
    ERROR MESSAGE: Removed too many insertions, header is now negative at position 27085688
    ERROR ------------------------------------------------------------------------------------------

    Any fix? - uma

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 6,804Administrator, GATK Developer admin

    The fix to this is in the very latest version, 2.6-5 (you are using -4). It was released just a few days ago.

    Geraldine Van der Auwera, PhD

  • BlueBlue Posts: 24Member

    I've just started getting the same error message (Removed too many insertions, header is now negative), having been using ReduceReads for several months on 2.4 and 2.6.

    I'm interested in the cause of the problem as well as the solution.

    The two changes in my protocol have been: 1) Use of Illumina Nextera library preparation method (instead of NEB-Next) 2) Using EA-Utils mcq-fasta to remove adaptor sequences. (Previously I used Scythe and Sickle to remove adaptors and poor bases.)

    I was wondering if either or both of these processes might be contributing to the problem.

    My command and error message below:

    Program Args: -T ReduceReads -R /home/b/bb/bb1/accessory_data_130613/Dmel_ref_seqs_051112/dmel-all-chromosome-r5.9.fasta -I /scratch/bioenv/bb1/tmp/read_mapping/RGillEA/RGillEA_CRSNR.bam -o /scratch/bioenv/bb1/tmp/read_mapping/RGillEA/RGillEA_r.bam INFO 17:39:07,759 HelpFormatter - Date/Time: 2013/09/10 17:39:07

    ERROR stack trace org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Removed too many insertions, header is now negative at position 7792698

    at org.broadinstitute.sting.gatk.walkers.compression.reducereads.HeaderElement.removeInsertionToTheRight(HeaderElement.java:210) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.actuallyUpdateHeaderForRead(SlidingWindow.java:1246) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.updateHeaderCounts(SlidingWindow.java:1162) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.removeFromHeader(SlidingWindow.java:1139) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.compressVariantRegion(SlidingWindow.java:742) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegion(SlidingWindow.java:835) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegions(SlidingWindow.java:884) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SlidingWindow.closeVariantRegions(SlidingWindow.java:850) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.SingleSampleCompressor.closeVariantRegions(SingleSampleCompressor.java:149) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.MultiSampleCompressor.closeVariantRegionsInAllSamples(MultiSampleCompressor.java:158) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.MultiSampleCompressor.addAlignment(MultiSampleCompressor.java:127) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReadsStash.compress(ReduceReadsStash.java:116) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReads.reduce(ReduceReads.java:477) at org.broadinstitute.sting.gatk.walkers.compression.reducereads.ReduceReads.reduce(ReduceReads.java:113) at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano$TraverseReadsReduce.apply(TraverseReadsNano.java:251) at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano$TraverseReadsReduce.apply(TraverseReadsNano.java:240) at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:279) at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245) at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano.traverse(TraverseReadsNano.java:102) at org.broadinstitute.sting.gatk.traversals.TraverseReadsNano.traverse(TraverseReadsNano.java:56) at org.broadinstitute.sting.gatk.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:108) at org.broadinstitute.sting.gatk.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:311) at org.broadinstitute.sting.gatk.CommandLineExecutable.execute(CommandLineExecutable.java:113) at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:245) at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:152) at org.broadinstitute.sting.gatk.CommandLineGATK.main(CommandLineGATK.java:91)

    ERROR ------------------------------------------------------------------------------------------
    ERROR A GATK RUNTIME ERROR has occurred (version 2.6-3-gdee51c4):
  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 6,804Administrator, GATK Developer admin

    As I recall the problem was related to read clipping, so it might be your adaptor trimming step that introduced the issue in your workflow. For exact details you'd have to look up the commit message of the fix on github. My recommendation would be to upgrade your version, or at least run RR from the latest version.

    Geraldine Van der Auwera, PhD

Sign In or Register to comment.