To celebrate the release of GATK 4.0, we are giving away free credits for running the GATK4 Best Practices pipelines in FireCloud, our secure online analysis portal. It’s first come first serve, so sign up now to claim your free credits worth $250. Sponsored by Google Cloud. Learn more at https://software.broadinstitute.org/firecloud/documentation/freecredits

HaplotypeCaller 4.beta.6 gVCF performance

Hi, ever since the 4.beta.4 release, I've noticed a significant increase in the memory requirements and execution time of HaplotypeCaller in gVCF mode. I tested the 4.beta.2 and 4.beta.6 version of HaplotypeCaller with a NA12878 BAM, aligned with BWA 0.7.13 with approximately 30x coverage. 4.beta.2 completed after roughly 5h with 2GB of memory, while 4.beta.6 completed after roughly 30h with 15GB of memory. 4.beta.6 failed with an out of memory exception when given less memory.

Both versions were ran with the same settings (--interval_set_rule UNION --genotyping_mode DISCOVERY --createOutputVariantIndex --emitRefConfidence GVCF) and parallelized on intervals from a custom BED file.

From my understanding of the release notes, the versions from 4.beta.4 onwards have a bug fix that corrects the results of HaplotypeCaller in gVCF mode. Is the performance difference to be expected?

Thank you,
Teodora

Best Answer

Answers

  • Hi @Geraldine_VdAuwera, that's good to hear. Thank you for the quick response.

  • Hi @Geraldine_VdAuwera , I started testing the full release of GATK 4. So far, I've noticed the same performance issues as with 4.beta.6. Is this behavior still expected?

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie

    No that’s not expected, can you please provide some details of what you are doing and what issues you observe?

  • Hi @Geraldine_VdAuwera , thanks for the quick response!

    We used CCLE WES FASTQs for the purpose of testing (6GB per paired end). We scattered the tool using a custom BED file with 86 intervals.

    4.0.0.0 failed due to memory with 2GB per job. It completed with 4GB per job. Each job lasted approximately 1h 10 min. 4.beta.2 completed with 2GB per job and each job lasted approximately 5m.

    This is in gVCF mode, with the same settings as described above. All input files, settings and hardware are the same between versions.

    Issue · Github
    by Sheila

    Issue Number
    4169
    State
    open
    Last Updated
    Assignee
    Array
    Milestone
    Array
  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie

    Out of curiosity, did you check the validity of the results you got with the beta.2?

  • We have tested the validity of a Whole Genome GATK 4.beta2 workflow (in gVCF mode, with VQSR) with a number of different library preps of HG001-HG005 GIAB samples. All of the scores were at expected levels, for example: HG001-50x SNP (precision=99.91, recall=99.75), INDEL (precision=99.40, recall=98.65), HG002-50x SNP (precision=99.90, recall=99.71), INDEL (precision=99.47, recall=98.54).
    Also, an intersection between the VCFs produced by 4.beta.2 and 4.0.0.0 (the same Whole Genome workflow configuration) using mentioned CCLE WES files produced 100 000 matching variants, while 200 were different.

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @teodora_aleksic
    Hi Teodora,

    Thanks for the information. I will pass this on to the team, and someone will get back to you soon.

    -Sheila

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @teodora_aleksic
    Hi again Teodora,

    Can you provide us with a test case to reproduce this? The developers say their own profiling shows that the 4.0 version of HaplotypeCaller is much faster than the later betas. Can you also try running with GATK3 and letting us know if GATK4 is faster?

    The 4.beta.2 version of HaplotypeCaller does NOT give correct output at all, while the 4.0 version gives output that is almost identical to GATK3. HaplotypeCaller in 4.beta.2 was not a complete version of the tool, so it is not appropriate to be comparing the release to it.

    -Sheila

  • Hi everyone,

    We found the reason why HaplotypeCaller 4.0.0.0 performed worse than 4.beta.2. We are scattering tools using intervals from a custom BED file. Before, each instance of HaplotypeCaller received a BAM file from ApplyBQSR that was produced using a specific interval, but not the interval itself. This worked for 4.beta.2, but was causing poor performance for 4.0.0.0. We now pass both the BAM and the interval to HaplotypeCaller 4.0.0.0 and it performs just as well as 4.beta.2 with the same amount of memory.

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @teodora_aleksic
    Hi Teodora,

    Thank you for reporting your findings :smile: I will pass this along to the team.

    -Sheila

Sign In or Register to comment.