Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.

Issues on FilterMutectCalls: Log10-probability must be 0 or less

jrlcloudjrlcloud BeijingMember
edited March 18 in Ask the GATK team

When I use FilterMutectCalls to Filter VCF resutls called by Mutect2, I got "Log10-probability must be 0 or less" Error on some of the VCF results (not all of them got the error message, maybe 10 out of 50 or something...), and it seems that the filtering process is done since I got the "org.broadinstitute.hellbender.tools.walkers.mutect.FilterMutectCalls done. Elapsed time: 0.54 minutes" message before the error alert, but I got no filtered vcf in my output directory.

Additional Info: the samples' vcf that got issues on FilterMutectCalls use the exact same experimental and bioinformatics approach with those samples of the same batch on which FilterMutectCalls works perfectly normal.

The detailed log is like this (PS: My GATK version is 4.1.0.0):

19:15:30.504 INFO FilterMutectCalls - Done initializing engine
19:15:30.573 INFO ProgressMeter - Starting traversal
19:15:30.574 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute
19:15:30.574 INFO FilterMutectCalls - Starting first pass through the variants
19:15:30.606 INFO FilterMutectCalls - Shutting down engine
[March 11, 2019 7:15:30 PM CST] org.broadinstitute.hellbender.tools.walkers.mutect.FilterMutectCalls done. Elapsed time: 0.54 minutes.
Runtime.totalMemory()=2032140288
java.lang.IllegalArgumentException: log10p: Log10-probability must be 0 or less
at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:724)
at org.broadinstitute.hellbender.utils.MathUtils.log10BinomialProbability(MathUtils.java:1031)
at org.broadinstitute.hellbender.utils.MathUtils.binomialProbability(MathUtils.java:1024)
at org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2FilteringEngine.applyContaminationFilter(Mutect2FilteringEngine.java:68)
at org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2FilteringEngine.calculateFilters(Mutect2FilteringEngine.java:518)
at org.broadinstitute.hellbender.tools.walkers.mutect.FilterMutectCalls.firstPassApply(FilterMutectCalls.java:130)
at org.broadinstitute.hellbender.engine.TwoPassVariantWalker.lambda$traverseVariants$0(TwoPassVariantWalker.java:76)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.broadinstitute.hellbender.engine.TwoPassVariantWalker.traverseVariants(TwoPassVariantWalker.java:74)
at org.broadinstitute.hellbender.engine.TwoPassVariantWalker.traverse(TwoPassVariantWalker.java:27)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:966)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:138)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
at org.broadinstitute.hellbender.Main.main(Main.java:291)

Could anyone please help to verify this?

Best Answers

Answers

  • jrlcloudjrlcloud BeijingMember
    edited March 18

    I move the additional info on the question above.

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭
    Accepted Answer

    I believe this has been fixed in the gatk master branch for a week or two. You could build master now or wait for the 4.1.1 release which is scheduled for Tuesday.

  • jrlcloudjrlcloud BeijingMember

    @davidben said:
    I believe this has been fixed in the gatk master branch for a week or two. You could build master now or wait for the 4.1.1 release which is scheduled for Tuesday.

    Great! Looking forward to the 4.1.1 release. Thanks for the clarification~

  • jrlcloudjrlcloud BeijingMember
    edited March 31

    @davidben said:
    I believe this has been fixed in the gatk master branch for a week or two. You could build master now or wait for the 4.1.1 release which is scheduled for Tuesday.

    I tried again with the 4.1.1 release, it seems that the same issue is still there, could you please help to check again?

    the detailed log is like this:

    java.lang.IllegalArgumentException: log10p: Log10-probability must be 0 or less
    at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:724)
    at org.broadinstitute.hellbender.utils.MathUtils.log10BinomialProbability(MathUtils.java:979)
    at org.broadinstitute.hellbender.utils.MathUtils.binomialProbability(MathUtils.java:972)
    at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.ContaminationFilter.calculateErrorProbability(ContaminationFilter.java:55)
    at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.Mutect2VariantFilter.errorProbability(Mutect2VariantFilter.java:15)
    at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.ErrorProbabilities.lambda$new$1(ErrorProbabilities.java:19)
    at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321)
    at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.ErrorProbabilities.(ErrorProbabilities.java:19)
    at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.Mutect2FilteringEngine.accumulateData(Mutect2FilteringEngine.java:136)
    at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.FilterMutectCalls.nthPassApply(FilterMutectCalls.java:140)
    at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.lambda$traverse$0(MultiplePassVariantWalker.java:31)
    at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.lambda$traverseVariants$1(MultiplePassVariantWalker.java:68)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.traverseVariants(MultiplePassVariantWalker.java:66)
    at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.traverse(MultiplePassVariantWalker.java:31)
    at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:984)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:138)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
    at org.broadinstitute.hellbender.Main.main(Main.java:291)

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @jrlcloud are you able to share your input unfiltered vcf for debugging?

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    Also @jrlcloud did you run the full M2 pipeline including CalculateContamination? If so, could you post the -contamination-table file input to FilterMutectCalls (the whole file is just two lines).

  • jrlcloudjrlcloud BeijingMember

    @davidben yes, I did run the CalculateContamination and use the contamination-table file as the input of the filtering process, I will sent you the unfiltered vcf through email for debugging later on~

  • jrlcloudjrlcloud BeijingMember

    @davidben Yes, it is indeed a small gene panel, and the FilterMutectCalls process finished all right without the contamination-table file as an input. So could you please give me a minimum panel size for a reliable CalculateContamination results ? And the unfiltered VCF was sent to your working email, please check, thanks !

  • jrlcloudjrlcloud BeijingMember

    @davidben Thanks again for the debugging and panel size recommendation, looking forward to see some exciting updates for panel data handling in the furture :)

  • micknudsenmicknudsen DenmarkMember ✭✭

    @davidben I still encounter this error in GATK v4.1.2. I believe it is caused by an NaN contamination estimate.

    sample  contamination   error
    MY_EXOME_SAMPLE  NaN     1.0
    

    I get the NaN both with and without the --matched-normal option in CalculateContamination.

    [May 27, 2019 11:52:46 PM CEST] org.broadinstitute.hellbender.tools.walkers.mutect.filtering.FilterMutectCalls done. Elapsed time: 0.06 minutes.
    Runtime.totalMemory()=1037959168
    java.lang.IllegalArgumentException: log10p: Log10-probability must be 0 or less
        at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:724)
        at org.broadinstitute.hellbender.utils.MathUtils.log10BinomialProbability(MathUtils.java:917)
        at org.broadinstitute.hellbender.utils.MathUtils.binomialProbability(MathUtils.java:910)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.ContaminationFilter.calculateErrorProbability(ContaminationFilter.java:56)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.Mutect2VariantFilter.errorProbability(Mutect2VariantFilter.java:15)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.ErrorProbabilities.lambda$new$1(ErrorProbabilities.java:19)
        at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1321)
        at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.ErrorProbabilities.<init>(ErrorProbabilities.java:19)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.Mutect2FilteringEngine.accumulateData(Mutect2FilteringEngine.java:141)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.FilterMutectCalls.nthPassApply(FilterMutectCalls.java:140)
        at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.lambda$traverse$0(MultiplePassVariantWalker.java:40)
        at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.lambda$traverseVariants$1(MultiplePassVariantWalker.java:77)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
        at java.util.Iterator.forEachRemaining(Iterator.java:116)
        at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
        at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.traverseVariants(MultiplePassVariantWalker.java:75)
        at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.traverse(MultiplePassVariantWalker.java:40)
        at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1039)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
        at org.broadinstitute.hellbender.Main.main(Main.java:291)
    Using GATK jar /home/michaelk/anaconda3/envs/moma-somatic-pipeline-v0.1.1/share/gatk4-4.1.2.0-1/gatk-package-4.1.2.0-local.jar
    
  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @jrlcloud The 4.1.2 release improves CalculateContamination for small panels.

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @micknudsen This might be fixed in a recent PR (not yet released). You could build the master branch of the GATK or download a jar from here: gs://broad-dsde-methods-davidben/gatk-builds/contamination-patch-5-27-2019.jar. Please let me know if this does not resolve the issue!

  • micknudsenmicknudsen DenmarkMember ✭✭

    @davidben Thanks! The patch resolves the issue. Looking forward to the next release.

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    Excellent. Thanks for letting me know!

  • alanhoylealanhoyle UNC Lineberger Member

    I have been developing a somatic variant-calling workflow and have been running into this issue as well. For testing, we've taken a few subjects that had targeted sequencing and filtered the reads down to just chr22 so we could run our development workflows over the course of an afternoon instead of more than a day. I look forward to a version that would allow me to use this small "toy" data set in addition to the full input.

  • pkhadkapkhadka Member

    @davidben I am trying to run Mutect2 on tumor-only mode on Firecloud and have been running into this exact same error. Can you please tell how can I build the master branch of GATK on Firecloud? It's running on 4.1.2.0 right now.

  • omw90omw90 Member
    edited August 1
    like @pkhadka I am also running into this same issue on the firecloud. @davidben any thoughts on how to fix?
  • pkhadkapkhadka Member

    @omw90 you can solve this problem by reverting to the 4.1.1.0 version of GATK. The problem is only in the newer version.

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @pkhadka @omw90 @alanhoyle The patch jar is now also at gs://gatk-best-practices/somatic-hg38/gatk-builds_contamination-patch-5-27-2019.jar. On Firecloud / Terra you should set the gatk_override input to this path. This works with the most recent gatk docker.

  • omw90omw90 Member

    @pkhadka said:
    @omw90 you can solve this problem by reverting to the 4.1.1.0 version of GATK. The problem is only in the newer version.

    @pkhadka is there a way to do this in Terra?

  • pkhadkapkhadka Member
    edited August 16

    @omw90 I am sure you can change the docker in Terra, but I'd suggest the fix by davidben (see above!)

    "@pkhadka @omw90 @alanhoyle The patch jar is now also at gs://gatk-best-practices/somatic-hg38/gatk-builds_contamination-patch-5-27-2019.jar. On Firecloud / Terra you should set the gatk_override input to this path. This works with the most recent gatk docker."

  • micknudsenmicknudsen DenmarkMember ✭✭

    @omw90 The bug has been fixed in GATK 4.1.3.0.

  • 29043594952904359495 Member

    @micknudsen , in 4.1.3.0, I still meet the same question

  • Tiffany_at_BroadTiffany_at_Broad Cambridge, MAMember, Administrator, Broadie, Moderator admin

    @2034359495 Are you saying you are still seeing the same issue in the latest version?

  • 29043594952904359495 Member

    @micknudsen ,thanks a lot.
    @Tiffany_at_Broad , yes, my panle is just 0.1M, I do not whethet it is because og this

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @2904359495 Would you mind uploading your output from GetPileupSummaries? For a small panel it should be a pretty small file.

  • 29043594952904359495 Member

    OK, I have sent you the google drive link, Please pay attention to information collection, thanks a lot

  • amjaddamjadd FinlandMember ✭✭

    @micknudsen said:
    @omw90 The bug has been fixed in GATK 4.1.3.0.

    I get the same error in GATK 4.1.3.0 not only with contamination but also with -ob-priors

  • bshifawbshifaw Member, Broadie, Moderator admin

    @amjadd ,

    To help with debugging the issue you may want to upload the files Davidb mentioned earlier in the thread. Such as your input unfiltered vcf, contamination-table file input to FilterMutectCalls, etc.

  • amjaddamjadd FinlandMember ✭✭

    @bshifaw Is there any instructions on how to upload files?

  • Tiffany_at_BroadTiffany_at_Broad Cambridge, MAMember, Administrator, Broadie, Moderator admin
  • 29043594952904359495 Member

    I meet this in 4.1.1.0,which not show in this version, but just in 4.1.2.0 and4= 4.1, due to privacy, I have sent the file to you @bshifaw thanks a lot

  • amjaddamjadd FinlandMember ✭✭

    Thank you @Tiffany_at_Broad
    bug_report_mutect2_obpriors.zip was uploaded.

  • 29043594952904359495 Member
    edited September 3

    I meet this in 4.1.1.0 the forst time,which not show in this version, but just in 4.1.2.0 and4.1.3.0 before, due to privacy, I have sent the file to you @bshifaw thanks a lot

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @2904359495 The changes in this pull request: https://github.com/broadinstitute/gatk/pull/6137 fixes the issue on your data, and hopefully for all small gene panels.

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    I get the same error in GATK 4.1.3.0 not only with contamination but also with -ob-priors

    @amjadd The fix is going in soon: https://github.com/broadinstitute/gatk/pull/6141

  • 29043594952904359495 Member

    @davidben , thanks a lot, very waiting for the new version of gatk

  • amjaddamjadd FinlandMember ✭✭

    @davidben Thank you very much.

Sign In or Register to comment.