Holiday Notice:
The Frontline Support team will be offline February 18 for President's Day but will be back February 19th. Thank you for your patience as we get to all of your questions!

GenotypeAndValidate error: Key callStatus found in VariantContext field INFO

Hi,

I am trying to run GenotypeAndValidate on 4 of my bam files. I get the following error. I am sure I do not have 'callStatus' in the info field of my 'alleles' vcf file. Thanks.

INFO 22:21:20,191 HelpFormatter - --------------------------------------------------------------------------------
INFO 22:21:20,194 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.0-0-g6bad1c6, Compiled 2014/03/06 06:30:35
INFO 22:21:20,194 HelpFormatter - Copyright (c) 2010 The Broad Institute
INFO 22:21:20,195 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk
INFO 22:21:20,201 HelpFormatter - Program Args: -R hs37d5.fasta -T GenotypeAndValidate -alleles variantCallsHC_filtered.vcf -I p1.clean.dedup.recal.bam -I p2.clean.dedup.recal.bam -I F.bam -I M.bam -o genotypeNvalidate.vcf -L variantCallsHC_filtered.vcf
INFO 22:21:20,205 HelpFormatter - Executing as * on Linux 2.6.32-220.13.1.el6.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.7.0_25-b15.
INFO 22:21:20,206 HelpFormatter - Date/Time: 2014/03/22 22:21:20
INFO 22:21:20,206 HelpFormatter - --------------------------------------------------------------------------------
INFO 22:21:20,207 HelpFormatter - --------------------------------------------------------------------------------
INFO 22:21:21,135 GenomeAnalysisEngine - Strictness is SILENT
INFO 22:21:21,329 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 1000
INFO 22:21:21,341 SAMDataSource$SAMReaders - Initializing SAMRecords in serial
INFO 22:21:21,407 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.06
INFO 22:21:42,545 IntervalUtils - Processing 135051 bp from intervals
INFO 22:21:42,659 GenomeAnalysisEngine - Preparing for traversal over 4 BAM files
INFO 22:21:43,598 GenomeAnalysisEngine - Done preparing for traversal
INFO 22:21:43,598 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING]
INFO 22:21:43,599 ProgressMeter - Location processed.sites runtime per.1M.sites completed total.runtime remaining

ERROR ------------------------------------------------------------------------------------------
ERROR stack trace

java.lang.IllegalStateException: Key callStatus found in VariantContext field INFO at 1:762273 but this key isn't defined in the VCFHeader. We require all VCFs to have complete VCF headers by default.
at org.broadinstitute.variant.vcf.VCFEncoder.fieldIsMissingFromHeaderError(VCFEncoder.java:173)
at org.broadinstitute.variant.vcf.VCFEncoder.encode(VCFEncoder.java:112)
at org.broadinstitute.variant.variantcontext.writer.VCFWriter.add(VCFWriter.java:214)
at org.broadinstitute.sting.gatk.io.storage.VariantContextWriterStorage.add(VariantContextWriterStorage.java:183)
at org.broadinstitute.sting.gatk.io.stubs.VariantContextWriterStub.add(VariantContextWriterStub.java:254)
at org.broadinstitute.sting.gatk.walkers.validation.GenotypeAndValidate.map(GenotypeAndValidate.java:493)
at org.broadinstitute.sting.gatk.walkers.validation.GenotypeAndValidate.map(GenotypeAndValidate.java:216)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:267)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:255)
at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:274)
at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano.traverse(TraverseLociNano.java:144)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano.traverse(TraverseLociNano.java:92)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano.traverse(TraverseLociNano.java:48)
at org.broadinstitute.sting.gatk.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:99)
at org.broadinstitute.sting.gatk.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:313)
at org.broadinstitute.sting.gatk.CommandLineExecutable.execute(CommandLineExecutable.java:121)
at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:248)
at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:155)
at org.broadinstitute.sting.gatk.CommandLineGATK.main(CommandLineGATK.java:107)

ERROR ------------------------------------------------------------------------------------------
ERROR A GATK RUNTIME ERROR has occurred (version 3.0-0-g6bad1c6):
ERROR
ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
ERROR If not, please post the error message, with stack trace, to the GATK forum.
ERROR Visit our website and forum for extensive documentation and answers to
ERROR commonly asked questions http://www.broadinstitute.org/gatk
ERROR
ERROR MESSAGE: Key callStatus found in VariantContext field INFO at 1:762273 but this key isn't defined in the VCFHeader. We require all VCFs to have complete VCF headers by default.
ERROR ------------------------------------------------------------------------------------------

Best Answer

Answers

  • annatannat Member
    edited October 2014

    Hi, Has this been fixed yet, as I'm getting this error at the moment (v3.2-2-gec30cee)?
    Alternatively, could you give me a line to add to the header that would work - I have tried a few but no luck!
    Cheers
    Anna

    Post edited by annat on
  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Hi @annat,

    I think this was fixed, but perhaps the fix didn't make it into the release. You can try the latest nightly build and see if that works.

    Have you grepped your file to check if the field occurs in your variants or not?

  • annatannat Member

    Hi @Geraldine, I tried the nightly build but still the same problem:

    ##### ERROR ------------------------------------------------------------------------------------------
    ##### ERROR stack trace 
    java.lang.IllegalStateException: Key callStatus found in VariantContext field INFO at chrM:3506 but this key isn't defined in the VCFHeader.  We require all VCFs to have complete VCF headers by default.
            at htsjdk.variant.vcf.VCFEncoder.fieldIsMissingFromHeaderError(VCFEncoder.java:176)
            at htsjdk.variant.vcf.VCFEncoder.encode(VCFEncoder.java:115)
            at htsjdk.variant.variantcontext.writer.VCFWriter.add(VCFWriter.java:221)
            at org.broadinstitute.gatk.engine.io.storage.VariantContextWriterStorage.add(VariantContextWriterStorage.java:182)
            at org.broadinstitute.gatk.engine.io.stubs.VariantContextWriterStub.add(VariantContextWriterStub.java:269)
            at org.broadinstitute.gatk.tools.walkers.validation.GenotypeAndValidate.map(GenotypeAndValidate.java:496)
            at org.broadinstitute.gatk.tools.walkers.validation.GenotypeAndValidate.map(GenotypeAndValidate.java:215)
            at org.broadinstitute.gatk.engine.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:267)
            at org.broadinstitute.gatk.engine.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:255)
            at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:274)
            at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
            at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:144)
            at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:92)
            at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:48)
            at org.broadinstitute.gatk.engine.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:99)
            at org.broadinstitute.gatk.engine.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:319)
            at org.broadinstitute.gatk.engine.CommandLineExecutable.execute(CommandLineExecutable.java:121)
            at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:248)
            at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:155)
            at org.broadinstitute.gatk.engine.CommandLineGATK.main(CommandLineGATK.java:107)
    ##### ERROR ------------------------------------------------------------------------------------------
    ##### ERROR A GATK RUNTIME ERROR has occurred (version nightly-2014-10-13-gf24cf57):
    ##### ERROR
    ##### ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
    ##### ERROR If not, please post the error message, with stack trace, to the GATK forum.
    ##### ERROR Visit our website and forum for extensive documentation and answers to 
    ##### ERROR commonly asked questions http://www.broadinstitute.org/gatk
    ##### ERROR
    ##### ERROR MESSAGE: Key callStatus found in VariantContext field INFO at chrM:3506 but this key isn't defined in the VCFHeader.  We require all VCFs to have complete VCF headers by default.
    ##### ERROR ------------------------------------------------------------------------------------------
    

    I cannot find this field in the vcf file.
    The VCF file was generated by GATK, so should comply.

  • annatannat Member

    Hi @Geraldine_VdAuwera‌
    Sorry to keep bothering you - do you know if this bug is going to be fixed?
    Cheers
    Anna

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Hi @annat,

    It seems we lost track of this issue, and that's why the fix never happened, sorry. I put in a fix just now; pending code review, it may be available in a nightly build as soon as tomorrow.

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    @annat This is fixed in the new release (v3.3).

  • Will_GilksWill_Gilks University of Sussex, UKMember ✭✭

    @annat @baesc @drramki

    I've got an incomplete VCF header problem as well (using gatk v3.4-46):
    Program Args: --analysis_type CombineGVCFs --reference_sequence ../../reference_sequences/dmel/v6.0/dm6.fa --variant rg_GVCF.list
    Returns:
    Key END found in VariantContext field INFO at chr2L:64003 but this key isn't defined in the VCFHeader.

    Previous reported error messages were.
    Key callStatus found in VariantContext field INFO at 1:762273 but this key isn't defined in the VCFHeader.
    Key callStatus found in VariantContext field INFO at chrM:3506 but this key isn't defined in the VCFHeader.
    Key AN found in VariantContext field INFO at Chr1:4000371 but this key isn't defined in the VCFHeader.
    gatkforums.broadinstitute.org/discussion/2331/select-variants-says-vcf-header-is-missing

  • SheilaSheila Broad InstituteMember, Broadie, Moderator admin

    @Will_Gilks
    Hi,

    It seems like this may be a bug in the latest versions of GATK. I have asked another user to submit a bug report. http://gatkforums.broadinstitute.org/discussion/5972/wonky-vcf-header#latest

    When she submits a bug report and I submit a bug report to the team, I will update there.

    -Sheila

  • says_anovasays_anova californiaMember

    Hi,

    I'm having a very similar issue, any updated advice?

    ERROR ------------------------------------------------------------------------------------------
    ERROR stack trace

    java.lang.IllegalStateException: Key found in VariantContext field INFO at chr1:786343 but this key isn't defined in the VCFHeader. We require all VCFs to have complete VCF headers by default.
    at htsjdk.variant.vcf.VCFEncoder.fieldIsMissingFromHeaderError(VCFEncoder.java:177)
    at htsjdk.variant.vcf.VCFEncoder.encode(VCFEncoder.java:115)
    at htsjdk.variant.variantcontext.writer.VCFWriter.add(VCFWriter.java:222)
    at org.broadinstitute.gatk.engine.io.storage.VariantContextWriterStorage.add(VariantContextWriterStorage.java:200)
    at org.broadinstitute.gatk.engine.io.stubs.VariantContextWriterStub.add(VariantContextWriterStub.java:272)
    at org.broadinstitute.gatk.tools.walkers.variantutils.SelectVariants.map(SelectVariants.java:851)
    at org.broadinstitute.gatk.tools.walkers.variantutils.SelectVariants.map(SelectVariants.java:309)
    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:267)
    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:255)
    at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:274)
    at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:144)
    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:92)
    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:48)
    at org.broadinstitute.gatk.engine.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:99)
    at org.broadinstitute.gatk.engine.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:315)
    at org.broadinstitute.gatk.engine.CommandLineExecutable.execute(CommandLineExecutable.java:121)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:248)
    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:155)
    at org.broadinstitute.gatk.engine.CommandLineGATK.main(CommandLineGATK.java:106)

    ERROR ------------------------------------------------------------------------------------------
    ERROR A GATK RUNTIME ERROR has occurred (version 3.5-0-g36282e4):
    ERROR
    ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.
    ERROR If not, please post the error message, with stack trace, to the GATK forum.
    ERROR Visit our website and forum for extensive documentation and answers to
    ERROR commonly asked questions http://www.broadinstitute.org/gatk
    ERROR
    ERROR MESSAGE: Key found in VariantContext field INFO at chr1:786343 but this key isn't defined in the VCFHeader. We require all VCFs to have complete VCF headers by default.
    ERROR ------------------------------------------------------------------------------------------
  • Will_GilksWill_Gilks University of Sussex, UKMember ✭✭

    @says_anova My problem was mostly down to typos. Check the input vcf (or vcf list) is complete, and code that generated it is correct. I was genotyping gvcfs but it seems you're doing something different.

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Since the error message doesn't actually give a name for the offending key I suspect some kind of formatting error. Can you post the VCF record for chr1:786343 ?

  • burkesquiresburkesquires Bethesda, MDMember

    Hi Geraldine,
    When I get the same error (using v3.4-0-g7e26428) here is the VCF record that is causing the problem:

    1 871269 . A C 368.47 PASS AC=1;AF=0.031;AN=32;BaseQRankSum=-17.406;DP=351;Dels=0.00;FS=92.803;HaplotypeScore=0.5138;InbreedingCoeff=-0.0332;MQ=59.30;MQ0=0;MQRankSum=2.373;QD=0.48;ReadPosRankSum=-11.798;SNPEFF_AMINO_ACID_CHANGE=R141;SNPEFF_CODON_CHANGE=cgA/cgC;SNPEFF_EFFECT=SYNONYMOUS_CODING;SNPEFF_EXON_ID=exon_1_871152_871276;SNPEFF_FUNCTIONAL_CLASS=SILENT;SNPEFF_GENE_BIOTYPE=protein_coding;SNPEFF_GENE_NAME=SAMD11;SNPEFF_IMPACT=LOW;SNPEFF_TRANSCRIPT_ID=ENST00000342066;VQSLOD=-8.167e-01;culprit=QD; GT:AD:DP:GQ:PL 0/0:34,0:34:99:0,102,1073 0/0:34,0:34:99:0,102,1064 0/0:37,0:37:99:0,108,1155 0/0:8,3:11:24:0,24,226 0/1:11,4:16:32:32,0,300 0/0:10,0:10:30:0,30,306 0/0:13,0:13:39:0,39,410 0/0:11,0:11:33:0,33,323 0/0:21,3:23:12:0,12,434 0/0:19,0:19:57:0,57,581 0/0:25,4:28:27:0,27,553 0/0:17,1:18:51:0,51,524 0/0:25,0:25:75:0,75,759 0/0:21,0:21:63:0,63,687 0/0:23,4:27:69:0,69,709 0/0:22,2:24:60:0,60,562

  • burkesquiresburkesquires Bethesda, MDMember

    The VCF record above is the first record. If I delete it and try running again I get the same error but on the second record, which is this:
    1 876499 rs4372192 A G 212847.01 PASS AC=31;AF=0.969;AN=32;BaseQRankSum=24.257;DB;DP=122;Dels=0.01;FS=9.968;HaplotypeScore=0.2515;InbreedingCoeff=0.0411;MQ=59.13;MQ0=0;MQRankSum=0.764;QD=30.32;ReadPosRankSum=2.139;SNPEFF_EFFECT=DOWNSTREAM;SNPEFF_FUNCTIONAL_CLASS=NONE;SNPEFF_GENE_BIOTYPE=processed_transcript;SNPEFF_GENE_NAME=NOC2L;SNPEFF_IMPACT=MODIFIER;SNPEFF_TRANSCRIPT_ID=ENST00000327044;VQSLOD=4.70;culprit=MQ; GT:AD:DP:GQ:PL 1/1:0,7:7:21:253,21,0 1/1:0,8:8:24:296,24,0 1/1:0,4:4:12:148,12,0 1/1:0,5:5:15:184,15,0 1/1:0,1:1:3:39,3,0 1/1:0,5:5:15:176,15,0 1/1:0,9:9:27:316,27,0 1/1:0,11:11:33:397,33,0 1/1:0,7:7:18:212,18,0 1/1:0,11:11:27:369,27,0 1/1:0,13:13:39:492,39,0 0/1:7,9:15:99:235,0,158 1/1:0,7:7:18:231,18,0 1/1:0,4:4:12:154,12,0 1/1:0,8:8:24:296,24,0 1/1:0,7:7:21:270,21,0

  • burkesquiresburkesquires Bethesda, MDMember

    The problem is I had removed vep data from a VCF file and inadvertently kept the trailing semicolon ";" before the GT:AD....info. removing the trailing semicolon fixed the problem. It may have seen teh semicolon and tab and two delimiters with no KEY in between.

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Glad to hear your problem is solved.

  • I'm having a similar issue running 4.0.2.1. I get the error message:

    java.lang.IllegalStateException: Key END found in VariantContext field INFO at 1:28344 but this key isn't defined in the VCFHeader. We require all VCFs to have complete VCF headers by default.

  • SheilaSheila Broad InstituteMember, Broadie, Moderator admin

    @forgacs1
    Hi,

    Can you post the exact command you ran? How did you generate the VCF that is throwing the error?

    Thanks,
    Sheila

Sign In or Register to comment.