We've moved!
This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!

[GATK] how OPTIMAL_F_SCORE is calculated in FilterMutectCalls

I am using GATK4.1.2.0 FilterMutectCalls after applying Mutect2 with default option. I found --threshold-strategy OPTIMAL_F_SCORE is the actual filtering threshold to judge PASS or other. How is the score being calculated in the tool? Considering multiple values like nlod, tlod etc?



  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin
    edited April 2019

    Hi @jh2663

    We are working on creating a mutect2 tutorial with the new changes to the tool. It will be out shortly. In the mean time here is a blurb from the tutorial explaning the New Filtering Strategy in FilterMutectCalls:

    FilterMutectCalls now filters based on a single quantity, the probability that a variant is a somatic mutation. Previously, different reasons why this might not be the case each had their own thresholds. We have removed parameters such as -normal-artifact-lod, -max-germline-posterior, -max-strand-artifact-probability, and -max-contamination-probability. Even the previously fundamental -tumor-lod is gone. Rather than replace these with a single threshold for the error probability, FilterMutectCalls replaces them with nothing at all. It automatically determines the threshold that optimizes the "F score," the harmonic mean of sensitivity and precision. In order to tweak results in favor of more sensitivity users may set -f-score-beta to a value greater than its default of 1 (beta is the relative weight of sensitivity versus precision in the harmonic mean). Setting it lower biases results toward greater precision.

    You can think of these changes as doing two things. Unifying filtering thresholds puts all the filters at the same place on a ROC curve of sensitivity vs precision. Previously, one threshold might be sacrificing a lot of sensitivity for a small gain in precision while another might be doing the opposite, the result being poor sensitivity and precision that fell below the potential ROC curve. Once we’re on that ROC curve, we achieve a good balance between sensitivity and precision by optimizing the F score.

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @jh2663 Thanks for using the new version! Here is our documentation: https://github.com/broadinstitute/gatk/blob/master/docs/mutect/mutect.pdf. The big picture is that FilterMutectCalls is fully Bayesian, so TLOD, NLOD and other things are just intermediate likelihoods the model uses to calculate the posterior probability that each call is an error. Once you have the posterior probability you can estimate the F score as in this Wikipedia article: https://en.wikipedia.org/wiki/F1_score.

Sign In or Register to comment.