The Frontline Support team will be slow to respond December 17-18 due to an institute-wide retreat and offline December 22- January 1, while the institute is closed. Thank you for your patience during these next few weeks. Happy Holidays!
MuTect2 LOD calculations
I am interested in how NLOD and TLOD are calculated in MuTect2. The MuTect2.java code (https://github.com/broadgsa/gatk-protected/blob/master/protected/gatk-tools-protected/src/main/java/org/broadinstitute/gatk/tools/walkers/cancer/m2/MuTect2.java#L856) leads me to a function calcGenotypeLikelihoodsOfRefVsAny(), in which when a non-ref is observed:
genotypeLikelihoods[AB] += Math.log10(fpobs + (1-f)pobs/3.0d);
However, my understanding is that the real likelihood should be
genotypeLikelihoods[AB] += Math.log10(fpobs + (1-f)(1-pobs)/3.0d);
Is this an error, or could you clarify how these likelihood values, NLOD, TLOD are calculated?
I have read the algorithm guide here https://software.broadinstitute.org/gatk/guide/article?id=4442, but couldn't find my answer.
I have also read the MuTect1 paper https://www.ncbi.nlm.nih.gov/pubmed/23396013, but I don't know if they are the same as in MuTect2.