GATK licensing moves to direct-through-Broad model -- read about it on the GATK blog

HaplotypeCaller takes too long

rcholicrcholic DenverPosts: 68Member

I'm running the latest GATK nightly build to process human exome-seq data (has 12 samples). It seemed be faster than the older version until I run the HaplotypeCaller. The run summary shows it will take 14 days to finish. I am wondering if there's anything in my below command: How to make it faster without losing data in the output?

java -Xmx10g -Djava.io.tmpdir=/temp/GATK_temp
-jar $CLASSPATH/GenomeAnalysisTK.jar \
-T HaplotypeCaller \
-R ../GATK_ref/hg19.fasta \
-I ./compressedbam.list \
-L ../GATK_ref/hg19knownGene_UCSC_sorted.bed \
-log ../GATK/VQSR/log/HaplotypeCaller_20131018.log \
-o ../GATK/VQSR/raw.snps_indels.vcf
Tagged:

Answers

  • pdexheimerpdexheimer Posts: 463Member, GATK Dev, DSDE Member mod

    Use Queue :)

    Failing that, split up your interval manually into n intervals, run n HC jobs in parallel, then combine the resulting VCFs at the end

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 7,781Administrator, GATK Dev admin

    As we've discussed previously on the forum (search is your friend), adjusting the minPruning and parallelizing (-nct and Queue scatter-gather) will give you the biggest wins.

    Geraldine Van der Auwera, PhD

  • rcholicrcholic DenverPosts: 68Member

    @Geraldine_VdAuwera said:
    As we've discussed previously on the forum (search is your friend), adjusting the minPruning and parallelizing (-nct and Queue scatter-gather) will give you the biggest wins.

    thanks, yes, I wold like to use Queue scatter-gather. I've been trying to find good example to run Hapolotypecaller with it, do you have some helpful script that I can look at?

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 7,781Administrator, GATK Dev admin

    Have a look at the presentations from Tuesday's workshop, hopefully they will help you get started with Queue. The slides are here:

    http://gatkforums.broadinstitute.org/discussion/3367/presentation-slides-for-the-oct-2013-workshop-are-online

    The videos of the talks will be online in a couple of weeks.

    Geraldine Van der Auwera, PhD

  • rcholicrcholic DenverPosts: 68Member

    If you have multiple vcf files output, how do you merge them exactly? thanks

    @pdexheimer said:
    Use Queue :)

    Failing that, split up your interval manually into n intervals, run n HC jobs in parallel, then combine the resulting VCFs at the end

  • pdexheimerpdexheimer Posts: 463Member, GATK Dev, DSDE Member mod

    CombineVariants

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 7,781Administrator, GATK Dev admin

    Or if you're running on separate intervals (not separate samples), you can get away with using CatVariants, which is much faster. It's what the Queue gatherer uses internally when you scatter-gather HC jobs.

    Geraldine Van der Auwera, PhD

  • rcholicrcholic DenverPosts: 68Member

    Let me make sure I understand this. After processing the separate intervals with HC, I should use CatVariants to combine the vcf files. Is this faster than CombineVariants? thanks Geraldine.

    @Geraldine_VdAuwera said:
    Or if you're running on separate intervals (not separate samples), you can get away with using CatVariants, which is much faster. It's what the Queue gatherer uses internally when you scatter-gather HC jobs.

  • Geraldine_VdAuweraGeraldine_VdAuwera Posts: 7,781Administrator, GATK Dev admin

    After processing the separate intervals with HC, I should use CatVariants to combine the vcf files.

    Yes, that's correct. It's faster than CombineVariants because it will just straight up concatenate the variants from the different runs, and won't check for overlaps or try to do any kind of fancy merging. The caveat is that you should only use this on VCFs generated for the same set of samples for different non-overlapping intervals.

    Geraldine Van der Auwera, PhD

Sign In or Register to comment.