We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!
Test-drive the GATK tools and Best Practices pipelines on Terra
Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
We will be out of the office for a Broad Institute event from Dec 10th to Dec 11th 2019. We will be back to monitor the GATK forum on Dec 12th 2019. In the meantime we encourage you to help out other community members with their queries.
Thank you for your patience!
Does Base Recalibrator require every contig to contain a known SNP or Indel?

Hi all,
Apologies if this seems like a duplication of previous questions but I have been unable to find an answer to my problem.
In the absence of any existing high quality SNPs upon which to base VQSR and BQSR, I have called and filtered SNPs using Haplotype Caller and am now looking to feed my HQ SNPs into BQSR. When running BaseRecalibrator, however, I receive a message saying that the contigs in my vcf file of known sites are incompatible with the contigs in my reference file. In the vcf file's header section, the contigs are the exact same length and in the exact same order as in the reference fasta file. There are 556 contigs lacking any HQ SNPs, so these contigs' names do not appear in the content of the vcf file. I assume that this is what is causing the program to fail.
I wondered if this was a bug or if it is intentional? I'm not sure how sorting the vcf, as is recommended, will help in a situation like this if not all contigs are represented in the body of the vcf?
Kind regards,
Ian
Best Answers
-
Geraldine_VdAuwera Cambridge, MA admin
Hi Ian, a couple of comments:
1. There is no requirement for the presence of known variants within any particular contig.
2. You seem to be making a lot of unnecessary work for yourself. Most if not all the operations you ran can be achieved without any splitting up and re-merging of your data. I'm fairly certain that the error you're getting is due to something that went wrong during that complicated processing. By simplifying your workflow and avoiding any kind of parsing or manipulation outside of GATK, you would have much better control and visibility into what's happening to your data. We can advise you if you're not sure which tools to use. -
Sheila Broad Institute admin
@ianw
Hi Ian,So, do you have a separate VCF for each of the genome intervals?
You can use VariantFiltration and SelectVariants to do what you want. I think this article will help with the commands.To get only sites that have passed your filters, you can use SelectVariants with -excludeFiltered.
I hope this helps.
-Sheila
Answers
@ianw
Hi Ian,
Can you tell us the exact commands you ran to get the final VCF of HQ SNPs? If you used the entire reference in your command, all the contigs from the reference should be present in the VCF header.
-Sheila
Hi @Sheila,
As far as I can tell, every single contig from the reference is present and correct in the header of the VCF. That's why I'm assuming it's because not every one of those contigs actually contains a HQ variant. The commands I've used up until this stage for SNPs are as follows:
I originally created intervals for the reference genome, with the contigs ordered by size.
After running GenotypeGCVFs, I ran SelectVariants for each interval of the genome for SNPs and indels (not shown but identical) separately, for example:
Then, for each interval, I identified parameters by which to identify HQ variants, and filtered accordingly using VariantFiltration:
Once for the SNPs and once for the indels (not shown but identical), I then concatenated the intervals before pulling out only those lines that either form the header of the VCF or contain a HQ variant:
It is the following command that then complains about incompatible contigs:
Any ideas will be very welcome, even if I've just made a silly mistake!!
Thanks,
Ian
1. There is no requirement for the presence of known variants within any particular contig.
2. You seem to be making a lot of unnecessary work for yourself. Most if not all the operations you ran can be achieved without any splitting up and re-merging of your data. I'm fairly certain that the error you're getting is due to something that went wrong during that complicated processing. By simplifying your workflow and avoiding any kind of parsing or manipulation outside of GATK, you would have much better control and visibility into what's happening to your data. We can advise you if you're not sure which tools to use.
Hi @Geraldine_VdAuwera,
Your first comment is good to know, thanks - that removes one concern
Regarding your second point, yes, some pointers on what I could change may be helpful, please. Unfortunately, I'd split the genome into intervals because of storage issues on the server I'm using. Also, apart from pulling out the desired lines from the VCF, I haven't used any non-GATK commands, although I suspect there was an option I missed to just output variants that 'PASS' the filters..?
Many thanks,
Ian
@ianw
Hi Ian,
So, do you have a separate VCF for each of the genome intervals?
You can use VariantFiltration and SelectVariants to do what you want. I think this article will help with the commands.
To get only sites that have passed your filters, you can use SelectVariants with -excludeFiltered.
I hope this helps.
-Sheila
Hi @Sheila,
I had one vcf per interval until the VariantFiltration step, after which I concatenated them using CatVariants. I used the article you provided when writing out my commands, but I didn't use SelectVariants afterwards to exclude filtered sites. I'm trying a second run of SelectVariants (so that my pipeline goes 'SelectVariants' -> 'VariantFiltration' -> 'SelectVariants') now to see if it was the step I performed outside of GATK that caused the issues. I'll let you know how I get on.
Thanks,
Ian
Hi @Sheila and @Geraldine_VdAuwera,
I passed the VariantFiltration output back to SelectVariants so that it could pull out only those sites that passed the filters. The output of SelectVariants has now been accepted as the input for BaseRecalibration, which is running quite happily as I type. Thanks for the help!
Ian
@ianw
Hi Ian,
Great news!
-Sheila