The current GATK version is 3.3-0

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Posts: 47Member
edited November 2012

Hello dear GATK Team,

when trying to run Haplotypecaller on my exome files prepared with ReduceReads i get the error stated below.
As you can see the newest GATK Version is used. Also UnifiedGenotyper does not produce any errors on te exact same data (90 SOLiD exomes creatted according to Best Practice v4).

##### ERROR ------------------------------------------------------------------------------------------
##### ERROR stack trace
org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Somehow the requested coordinate is not covered by the read. Too many deletions?
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR A GATK RUNTIME ERROR has occurred (version 2.2-3-gde33222):
##### ERROR
##### ERROR Please visit the wiki to see if this is a known problem
##### ERROR If not, please post the error, with stack trace, to the GATK forum
##### ERROR
##### ERROR MESSAGE: Somehow the requested coordinate is not covered by the read. Too many deletions?
##### ERROR ------------------------------------------------------------------------------------------


The Command line used (abbreviated):

java -Xmx30g -jar /home/common/GenomeAnalysisTK-2.2-3/GenomeAnalysisTK.jar \
-R /home/common/hg19/ucschg19/ucsc.hg19.fasta \
-T HaplotypeCaller \
--dbsnp /home/common/hg19/dbsnp_135.hg19.vcf \
-o 93Ind_ped_reduced_HC_snps.raw.vcf \
-ped familys.ped \
--pedigreeValidationType SILENT \
-stand_call_conf 20.0 \
-stand_emit_conf 10.0

Post edited by Geraldine_VdAuwera on
Tagged:

• Posts: 122GATK Developer mod

Hi there,

Thanks for all your help with this issue. We believe this is fixed in the latest internal development version of the GATK and will be pushed out with the release of version 2.3

Thanks!

No ETA yet for 2.3 but on average we release every 6 weeks, and we last released 2.5 weeks ago...

Geraldine Van der Auwera, PhD

• Posts: 122GATK Developer mod

Good news. This is now fixed in our internal development version. You can start using our (unsupported) nightly builds to get the fix or wait for it to show up with the release of version 2.6 of the GATK.

Thanks for all your help with this.

• Posts: 122GATK Developer mod

Hi there,

That is a strange error to get. Are you able to narrow down the problem to a certain region of your input data file?

Would you be able to use PrintReads to extract an interval of your BAM file that reproduces the error and upload it to our FTP server?

Thanks!

• Posts: 47Member

I still could not narrow down the cause of the error by trying to run less samples and using a target intevall file.
By assiging the "--debug" parameter I get this before the Haploytpecaller crashes with the above message:

Found consecutive biallelic events with R^2 = 0.0521
-- [VC HC2 @ chr1:721362 Q. of type=INDEL alleles=[C*, CAAAAAGTA] attr={} GT=[]
-- [VC HC1 @ chr1:721368 Q. of type=SNP alleles=[T*, C] attr={} GT=[]
Found consecutive biallelic events with R^2 = 0.0069
-- [VC HC1 @ chr1:721368 Q. of type=SNP alleles=[T*, C] attr={} GT=[]
-- [VC HC4 @ chr1:721376 Q. of type=SNP alleles=[A*, G] attr={} GT=[]
Found consecutive biallelic events with R^2 = 0.0152
-- [VC HC9 @ chr1:721436 Q. of type=SNP alleles=[A*, G] attr={} GT=[]
-- [VC HC7 @ chr1:721446-721447 Q. of type=INDEL alleles=[CT*, C] attr={} GT=[]
Found consecutive biallelic events with R^2 = 0.0331
-- [VC HC7 @ chr1:721446-721447 Q. of type=INDEL alleles=[CT*, C] attr={} GT=[]
-- [VC HC3 @ chr1:721450 Q. of type=SNP alleles=[G*, A] attr={} GT=[]
Found consecutive biallelic events with R^2 = 0.0331
-- [VC HC3 @ chr1:721450 Q. of type=SNP alleles=[G*, A] attr={} GT=[]
-- [VC HC6 @ chr1:721454 Q. of type=SNP alleles=[C*, T] attr={} GT=[]
Found consecutive biallelic events with R^2 = 0.0331
-- [VC HC6 @ chr1:721454 Q. of type=SNP alleles=[C*, T] attr={} GT=[]
-- [VC HC5 @ chr1:721461 Q. of type=SNP alleles=[T*, C] attr={} GT=[]
Genotyping event at 721362 with alleles = [C*, CAAAAAGTA]
Genotyping event at 721368 with alleles = [T*, C]
Genotyping event at 721376 with alleles = [A*, G]
Genotyping event at 721436 with alleles = [A*, G]
Genotyping event at 721446 with alleles = [CT*, C]
Genotyping event at 721450 with alleles = [G*, A]
Genotyping event at 721454 with alleles = [C*, T]
Genotyping event at 721461 with alleles = [T*, C]
[VC UG_call @ chr1:721446-721447 Q43.44 of type=INDEL alleles=[CT*, C] attr={MLEAC=[1], MLEAF=[0.015151515151515152]}

I will try running each sample separatly with the HaplotypeCaller to rule out errors in the reduced BAMs and then merge the BAMs and upload the part around the above error.
Would that be of any help?

• Posts: 122GATK Developer mod
edited November 2012

Hi there,

It looks like you narrowed it down to the region around chr1:721446-721447.

If you run your command with, for example, -L chr1:721346-721546 does it reproduce the error?

Thanks for your help with this,

Post edited by rpoplin on
• Posts: 47Member

Yes is is reproducable with the -L chr1:721346-721546 command set.

But the error seems to occur multiple times. Running on Chromosome 2 yields an error after:

[VC UG_call @ chr2:234130 Q9769.92 of type=SNP alleles=[T*, G] attr={MLEAC=[37], MLEAF=[0.2890625]}
[VC UG_call @ chr2:234148-234149 Q69.86 of type=INDEL alleles=[TA*, T] attr={MLEAC=[1], MLEAF=[0.007575757575757576]}.

• Posts: 122GATK Developer mod

Well, let's do one at a time. Can you use PrintReads with the interval above to excise out the region and post the bam to our FTP server (instructions above).

Thanks!

• Posts: 47Member
edited November 2012

Hey,

I just uploaded the file to the ftp-server in the directory with my username.
Checked it in IGV but seems ok to me...

Also uploaded the part from Chromosome 2.
The error seems to occur at possitions with the "DEL" flag set and the value of this beeing >1.

Post edited by BerntPopp on
• Posts: 47Member

Also:
Running each sample separately on the above interval results in no errors and one individual is called with a 1bp deletion.

• Posts: 122GATK Developer mod
edited November 2012

Hmmm, I'm not able to reproduce the error with your files. I'm using this command line:

java -jar GenomeAnalysisTK.jar -T HaplotypeCaller -I 93Ind_chr1interval.ontarget.MarkDups.nRG.reor.Real.Recal.reduced.bam -R ucsc.hg19.fasta -L chr1:721346-721546 -debug -stand_call_conf 20.0 -stand_emit_conf 10.0

I wonder, have you tried running it with the latest version of the GATK?

Thanks,

Post edited by rpoplin on
• Posts: 47Member
edited November 2012

Strange, I also cant reproduce the error on the merged interval .bam.
Also the error on chromosome 1 now is at a other location, chr2 still the same when run with the unmerged bams.

I made a new, bigger interval file after merging the reduced bams (which did not help either) and uploaded it again.

Command line run with this inervall file:

java -Xmx16g -jar GenomeAnalysisTK.jar \
-R ucsc.hg19.fasta \
-T HaplotypeCaller \
-I 93Ind_chr1interval2.ontarget.MarkDups.nRG.reor.Real.Recal.reduced.bam \
-o 93Ind_ped_reduced_HC_snps.raw.vcf \
-L chr1 \
-stand_call_conf 20.0 \
-stand_emit_conf 10.0 \
-debug

produces this error:

[VC UG_call @ chr1:69453 Q51.38 of type=SNP alleles=[G*, A] attr={MLEAC=[4], MLEAF=[0.047619047619047616]}
[VC UG_call @ chr1:69455-69456 Q110.35 of type=MNP alleles=[CC*, AA] attr={MLEAC=[6], MLEAF=[0.061224489795918366]}
INFO 20:32:04,167 GATKRunReport - Uploaded run statistics report to AWS S3

ERROR stack trace

org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Somehow the requested coordinate is not covered by the read. Too many deletions?

ERROR ------------------------------------------------------------------------------------------

Can you reproduce this?
(Also could running the newest precompiled version alongside the older from surce compiled version on the server somehow interfer?)

Thanks a lot for your patient help!

Post edited by BerntPopp on
• Posts: 47Member
edited November 2012

I just tested the above command line on my private machine (GATK 2.2-8) with Fedora 17 and OpenJDK 1.7 and still get the exact same error. So it is reproducable and does not represent a faulty configuration on the CentOS Server.
Next i will try running it on Sun/Oracle Java JDK but i dont think that will do the trick eather.

UPDATE: same error with Java(TM) SE Runtime Environment (build 1.7.0_09-b04) on fedora 17.

Post edited by BerntPopp on
• Posts: 122GATK Developer mod

Hi there,

Thanks for all your help with this issue. We believe this is fixed in the latest internal development version of the GATK and will be pushed out with the release of version 2.3

Thanks!

• Posts: 47Member

Great!
Do you have a guess when 2.3 will be released?
Can't wait to try Haplotypecaller

No ETA yet for 2.3 but on average we release every 6 weeks, and we last released 2.5 weeks ago...

Geraldine Van der Auwera, PhD

• Posts: 6Member

Judging from BerntPopp's 11/19 post, it looks like he was informed that this is fixed in development (although I cannot see a comment from the GATK team to that effect).

If it hasn't been figured out yet, I can also send a small dataset (3 individuals, 8 reads after ReduceReads) that gives the same error (also with ReduceReads and HaplotypeCaller being run under version 2.2-8-gec077cd).

I would also suggest that (now that outside users do not have access to the full source code) more informative error messages are of even greater importance. If the exception were to include the read name (or even location and/or CIGAR) then it would be easier to isolate read(s) that lead to a problem. While not as good as being able to trace through the code, it would at least make it easier to make an example dataset that causes a problem.

• Posts: 459Member, GSA Collaborator ✭✭✭✭

Just a side note - the responses from the GATK team got promoted to answers and thus removed from the flow of the conversation (they're in green at the top). You have to read the thread and reconstruct their position in your head...

The alternative approach is what GetSatisfaction did, where the answers get duplicated at the top of the post. Neither option is particularly satisfying, but there it is

We're aware that it's a bit confusing when the "best answer" gets taken out of the flow of conversation. We'd prefer to promote a duplicate and keep the original answer in flow (like the GetSatisfaction system) but as far as we know this is not currently possible in Vanilla Forums. But they just featured us on their blog as a "great example of Q&A" so maybe we can use our newfound fame (hah) to pressure them to implement this as a feature...

Geraldine Van der Auwera, PhD

• Posts: 3Member

Hi.
Looks like this error is still with us in release version 2.3-5-g49ed93c.
I am getting this error when I run HaplotypeCaller on reduced bam:

ERROR stack trace

org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Somehow the requested coordinate is not covered by the read. Too many deletions?

ERROR ------------------------------------------------------------------------------------------

Hi @mbiob, we'll need you to upload a BAM snippet for testing so we can track down what's happening.

Geraldine Van der Auwera, PhD

• Posts: 3Member

Hi, and thanks for your respons!
OK.. so I have uploaded a zip archive, "mbiob_HC_rr_error.zip" containing: snippet.bam, snippet.bai, "command_line.txt", "Error.txt" and the bed file I used to constrict the region. I have tested and reproduced the error. Hope you can figure out what is going on, UG runs fine on the same bam files (reduced reads).

Thanks for the great example data! We are implementing a fix that will be available in the next GATK release (2.4, due out at the end of the month).

Eric Banks, PhD -- Senior Group Leader, MPG Analysis, Broad Institute of Harvard and MIT

• Posts: 47Member

@ebanks said:
Thanks for the great example data! We are implementing a fix that will be available in the next GATK release (2.4, due out at the end of the month).

Any release date yet? I'm also still struggling with this error

Cheers

Bernt

Sorry Bernt, I'm about to send out a notice that the release has been delayed. ETA is going to be 18 Feb at the earliest.

Geraldine Van der Auwera, PhD

• Posts: 59Member
edited February 2013

Hi,

I am having similar problems with haplotype caller although unifiedgenotyper seems to run fine. Here is the error output from GATK and the command.

I am using the latest version of GATK 2.3.9. I think there is no fix available as of yet for this problem.


java -Xmx10g -jar /u1/tools/public/GATK/GenomeAnalysisTK.jar -T HaplotypeCaller -R ../GATK_analysis/Soybean_ref_genome.fasta -I all.list -o haplotyp_caller_out.vcf
INFO 14:50:33,165 HelpFormatter - --------------------------------------------------------------------------------
INFO 14:50:33,168 HelpFormatter - The Genome Analysis Toolkit (GATK) v2.3-9-ge5ebf34, Compiled 2013/01/11 22:43:14
INFO 14:50:33,168 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk
INFO 14:50:33,173 HelpFormatter - Program Args: -T HaplotypeCaller -R ../GATK_analysis/Soybean_ref_genome.fasta -I all.list -o haplotyp_caller_out.vcf
INFO 14:50:33,173 HelpFormatter - Date/Time: 2013/02/25 14:50:33
INFO 14:50:33,173 HelpFormatter - --------------------------------------------------------------------------------
INFO 14:50:33,173 HelpFormatter - --------------------------------------------------------------------------------
INFO 14:50:33,200 GenomeAnalysisEngine - Strictness is SILENT
INFO 14:50:33,877 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 1000, Using the new downsampling implementation
INFO 14:50:33,884 SAMDataSource$SAMReaders - Initializing SAMRecords in serial INFO 14:50:34,506 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.61
INFO 14:50:34,703 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING]
INFO 14:50:34,703 ProgressMeter - Location processed.active regions runtime per.1M.active regions completed total.runtime remaining
INFO 14:51:04,706 ProgressMeter - Gm01:32768 1.64e+04 30.0 s 30.5 m 0.0% 10.3 d 10.3 d
INFO 14:52:04,708 ProgressMeter - Gm01:32768 1.64e+04 90.0 s 91.6 m 0.0% 4.4 w 4.4 w
INFO 14:52:25,284 GATKRunReport - Uploaded run statistics report to AWS S3

ERROR stack trace

org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Somehow the requested coordinate is not covered by the read. Too many deletions?

ERROR ------------------------------------------------------------------------------------------

Post edited by smk_84 on

The fix is in version 2.4 which should be released tonight if all goes well.

Geraldine Van der Auwera, PhD

• Posts: 59Member
edited March 2013

I am getting the following error when running haplotype caller ( GATK version 2.4-3

RestStorageService - Error Response: PUT '/GATK_Run_Reports/E5CIA8cJC5vdUpqEaMuOc4I1fVRwb51z.report.xml.gz' -- ResponseCode: 403, ResponseStatus: Forbidden, Request Headers: [Content-Length: 813, Content-MD5: sTZf6T0I
/Cb9UR2mMYmyrg==, Content-Type: application/octet-stream, x-amz-meta-md5-hash: b1365fe93d08fc26fd511da63189b2ae, Date: Mon, 11 Mar 2013 02:16:17 GMT

Post edited by smk_84 on

@smk_84, this error is unrelated to HaplotypeCaller, it's a problem with the Phone Home feature. It looks like the run report couldn't be uploaded. Can you confirm whether the GATK run completed successfully or not? If you are running GATK offline or behind a firewall, you may need to request a key to disable th ephone home feature. See this article for more details:

Geraldine Van der Auwera, PhD

• Posts: 59Member

No the GATK run did not complete successfully it crashed out after giving this error

@smk_84, you should try running again, and if the same error occurs you will need to apply for a GATK key using the link I posted earlier.

Geraldine Van der Auwera, PhD

• Posts: 17Member

Hi, I got the same error with HaplotypeCaller the latest release of GATK:

##### ERROR ------------------------------------------------------------------------------------------
##### ERROR stack trace
org.broadinstitute.sting.utils.exceptions.ReviewedStingException: Somehow the requested coordinate is not covered by the read. Too many deletions?
##### ERROR ------------------------------------------------------------------------------------------
##### ERROR A GATK RUNTIME ERROR has occurred (version 2.5-2-gf57256b):
##### ERROR
##### ERROR Please check the documentation guide to see if this is a known problem
##### ERROR If not, please post the error, with stack trace, to the GATK forum
##### ERROR
##### ERROR MESSAGE: Somehow the requested coordinate is not covered by the read. Too many deletions?
##### ERROR ------------------------------------------------------------------------------------------


I have already followed the best practice approach (except that maybe I have used GSNAP for alignment) so I am not sure whether if that is a problem due to alignment or not. One thing to note though, with previous version of GATK (1.4), I remember I have to use the -badCigar option for Unified genotyper due to a cigar having both I and D next to each other. Not sure if that is the same problem though...

• Posts: 122GATK Developer mod

Hi there,

Thanks for the information. Would you be able to upload a file snippet that reproduces the error so we can debug this. A command line would be helpful. Detailed instructions here: http://www.broadinstitute.org/gatk/guide/article?id=1894 - See more at: http://gatkforums.broadinstitute.org/discussion/comment/5902#Comment_5902

Thanks,

• Posts: 17Member

Hi @rpoplin,

I have uploaded the files to the ftp server using the file name: shingwan_gatk_bug_report.zip.
One thing to note though, when I use those samples separately, I got no error. It is only when I used them together did I got this error (In fact, I was originally handling 32 samples and these two samples alone were sufficient to generate the error....)

Sam

• Posts: 122GATK Developer mod

Hi there,

Thanks so much for the data that reproduces the problem. We have been able to replicate the issue here and are currently working on a fix for the next release. I'll send an update here when the fix is in place.

Thanks,

• Posts: 122GATK Developer mod