Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office on November 11th and 13th 2019, due to the U.S. holiday(Veteran's day) and due to a team event(Nov 13th). We will return to monitoring the GATK forum on November 12th and 14th respectively. Thank you for your patience.

IllegalArgumentException: beta must be greater than 0 in FilterMutectCalls

Hi there,

I have a simulated dataset of related samples and currently running Mutect2 on it (10 tumor samples WGS with 130x)
I managed to run everything through and now FilterMutectCalls crashes after the first pass through the variants with

[October 1, 2019 12:16:16 PM UTC] org.broadinstitute.hellbender.tools.walkers.mutect.filtering.FilterMutectCalls done. Elapsed time: 370.68 minutes.
Runtime.totalMemory()=20597702656
java.lang.IllegalArgumentException: beta must be greater than 0 but got -87566.7500301585
        at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:724)
        at org.broadinstitute.hellbender.tools.walkers.readorientation.BetaDistributionShape.<init>(BetaDistributionShape.java:14)
        at org.broadinstitute.hellbender.tools.walkers.mutect.clustering.BinomialCluster.getFuzzyBinomial(BinomialCluster.java:42)
        at org.broadinstitute.hellbender.tools.walkers.mutect.clustering.BinomialCluster.learn(BinomialCluster.java:33)
        at org.broadinstitute.hellbender.tools.walkers.mutect.clustering.SomaticClusteringModel.lambda$learnAndClearAccumulatedData$7(SomaticClusteringModel.java:131)
        at org.broadinstitute.hellbender.utils.IndexRange.forEach(IndexRange.java:116)
        at org.broadinstitute.hellbender.tools.walkers.mutect.clustering.SomaticClusteringModel.learnAndClearAccumulatedData(SomaticClusteringModel.java:131)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.Mutect2FilteringEngine.learnParameters(Mutect2FilteringEngine.java:156)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.FilterMutectCalls.afterNthPass(FilterMutectCalls.java:151)
        at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.traverse(MultiplePassVariantWalker.java:44)
        at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1039)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
        at org.broadinstitute.hellbender.Main.main(Main.java:291)
Using GATK jar /gatk/gatk-package-4.1.2.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /gatk/gatk-package-4.1.2.0-local.jar FilterMutectCalls --contamination-table /cromwell-executions/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/1230118915/generated-3208ebe8-e3ef-11e9-91de-005056b01e3e --tumor-segmentation /cromwell-executions/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/1230118915/generated-3208e648-e3ef-11e9-91de-005056b01e3e --stats /cromwell-executions/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/61973814/generated-58e77956-db7e-11e9-9da2-005056b01e3e.txt --orientation-bias-artifact-priors /cromwell-executions/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/-1768654832/generated-58e75278-db7e-11e9-9da2-005056b01e3e.tar.gz -V /cromwell-executions/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/164276910/generated-58e6fad0-db7e-11e9-9da2-005056b01e3e.vcf.gz -R /cromwell-executions/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/1500471319/human_g1k_v37.fasta -O generated-32095ea2-e3ef-11e9-91de-005056b01e3e.vcf.gz

I do not have any idea how to work around this.
Any suggestions?

Issue · Github
by bshifaw

Issue Number
6202
State
open
Last Updated
Assignee
Array

Answers

  • bshifawbshifaw Member, Broadie, Moderator admin

    There might be something wrong with one of your input files as suggested in this thread. Try running ValidateVariants on your VCF. How were the orientation-bias-artifact-priors and stat files generated?

  • SHollizeckSHollizeck Member

    Hey,

    thank you for your input I did validate the variants and no error came up

    Running:
        java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -jar /config/binaries/gatk/4.0/gatk-package-4.0.0.0-local.jar ValidateVariants -R /data/reference/indexes/human/hg19/fasta/Homo_sapiens.GRCh37.73.dna.toplevel.fa -V /researchers/sebastian.hollizeck/cromwell-execution/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/164276910/generated-58e6fad0-db7e-11e9-9da2-005056b01e3e.vcf.gz
    10:24:34.167 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/config/binaries/gatk/4.0/gatk-package-4.0.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
    10:24:34.674 INFO  ValidateVariants - ------------------------------------------------------------
    10:24:34.674 INFO  ValidateVariants - The Genome Analysis Toolkit (GATK) v4.0.0.0
    10:24:34.674 INFO  ValidateVariants - For support and documentation go to https://software.broadinstitute.org/gatk/
    10:24:34.675 INFO  ValidateVariants - Executing as [email protected] on Linux v3.10.0-957.10.1.el7.x86_64 amd64
    10:24:34.675 INFO  ValidateVariants - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_201-b09
    10:24:34.675 INFO  ValidateVariants - Start Date/Time: 7 October 2019 10:24:33 AM
    10:24:34.676 INFO  ValidateVariants - ------------------------------------------------------------
    10:24:34.676 INFO  ValidateVariants - ------------------------------------------------------------
    10:24:34.676 INFO  ValidateVariants - HTSJDK Version: 2.13.2
    10:24:34.676 INFO  ValidateVariants - Picard Version: 2.17.2
    10:24:34.676 INFO  ValidateVariants - HTSJDK Defaults.COMPRESSION_LEVEL : 1
    10:24:34.677 INFO  ValidateVariants - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    10:24:34.677 INFO  ValidateVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    10:24:34.677 INFO  ValidateVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    10:24:34.677 INFO  ValidateVariants - Deflater: IntelDeflater
    10:24:34.677 INFO  ValidateVariants - Inflater: IntelInflater
    10:24:34.677 INFO  ValidateVariants - GCS max retries/reopens: 20
    10:24:34.677 INFO  ValidateVariants - Using google-cloud-java patch 6d11bef1c81f885c26b2b56c8616b7a705171e4f from https://github.com/droazen/google-cloud-java/tree/dr_all_nio_fixes
    10:24:34.678 INFO  ValidateVariants - Initializing engine
    10:24:35.339 INFO  FeatureManager - Using codec VCFCodec to read file file:///researchers/sebastian.hollizeck/cromwell-execution/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/164276910/generated-58e6fad0-db7e-11e9-9da2-005056b01e3e.vcf.gz
    10:24:35.685 INFO  ValidateVariants - Done initializing engine
    10:24:35.686 INFO  ProgressMeter - Starting traversal
    10:24:35.686 INFO  ProgressMeter -        Current Locus  Elapsed Minutes    Variants Processed  Variants/Minute
    10:24:45.696 INFO  ProgressMeter -           1:92616453              0.2                990000        5935844.9
    10:24:55.695 INFO  ProgressMeter -          1:218466217              0.3               2084000        6249187.9
    10:25:05.698 INFO  ProgressMeter -           2:69957416              0.5               3172000        6341674.7
    10:25:15.699 INFO  ProgressMeter -          2:179300110              0.7               4290000        6432909.3
    10:25:25.701 INFO  ProgressMeter -           3:39718614              0.8               5414000        6494981.4
    10:25:35.706 INFO  ProgressMeter -          3:148270181              1.0               6561000        6558813.7
    10:25:45.710 INFO  ProgressMeter -           4:55973457              1.2               7672000        6573746.1
    10:25:55.712 INFO  ProgressMeter -          4:161889731              1.3               8821000        6613683.2
    10:26:05.718 INFO  ProgressMeter -           5:80600174              1.5               9966000        6641638.5
    10:26:15.725 INFO  ProgressMeter -            6:5665168              1.7              11112000        6664600.8
    10:26:25.733 INFO  ProgressMeter -          6:108210317              1.8              12190000        6646311.5
    10:26:35.736 INFO  ProgressMeter -           7:43193096              2.0              13337000        6665722.6
    10:26:45.737 INFO  ProgressMeter -          7:153939552              2.2              14485000        6682814.3
    10:26:55.738 INFO  ProgressMeter -          8:104298444              2.3              15629000        6695703.7
    10:27:05.738 INFO  ProgressMeter -           9:89555733              2.5              16774000        6707319.5
    10:27:15.738 INFO  ProgressMeter -          10:54670185              2.7              17862000        6696115.6
    10:27:25.741 INFO  ProgressMeter -          11:25407756              2.8              19011000        6707634.0
    10:27:35.742 INFO  ProgressMeter -         11:134936522              3.0              20158000        6717243.5
    10:27:45.745 INFO  ProgressMeter -         12:107501676              3.2              21293000        6722053.3
    10:27:55.745 INFO  ProgressMeter -          13:96626860              3.3              22416000        6722850.4
    10:28:05.746 INFO  ProgressMeter -         14:104817244              3.5              23547000        6725824.6
    10:28:15.748 INFO  ProgressMeter -          16:22251753              3.7              24672000        6726831.5
    10:28:25.757 INFO  ProgressMeter -          17:43350651              3.8              25709000        6704626.0
    10:28:35.768 INFO  ProgressMeter -          18:70655307              4.0              26848000        6709735.5
    10:28:45.775 INFO  ProgressMeter -          20:40453412              4.2              27934000        6701774.2
    10:28:55.779 INFO  ProgressMeter -            X:9544732              4.3              29026000        6695912.6
    10:29:05.782 INFO  ProgressMeter -          X:119197556              4.5              30160000        6699864.9
    10:29:11.342 INFO  ProgressMeter -     GL000248.1:21736              4.6              30768117        6697092.5
    10:29:11.342 INFO  ProgressMeter - Traversal complete. Processed 30768117 total variants in 4.6 minutes.
    10:29:11.342 INFO  ValidateVariants - Shutting down engine
    [7 October 2019 10:29:11 AM] org.broadinstitute.hellbender.tools.walkers.variantutils.ValidateVariants done. Elapsed time: 4.62 minutes.
    

    The stats file was created by mutect2 for each shard and then joined with MergeMutectStats
    similar the read orientation model was built with the f1r2 files from all shards.

    and like I said, this error only comes after the first pass of filtermutectCalls completed.

  • bshifawbshifaw Member, Broadie, Moderator admin

    Hey @SHollizeck ,

    This might be a bug, I've created an issue ticket. It would help us greatly if you could pass along the files for us to recreate the error. Instructions

  • SHollizeckSHollizeck Member

    Hi @bshifaw

    I uploaded the tar ball as SHollizeckUploadGATK-2.tar.gz as instructed.
    Due to a connection reset SHollizeckUploadGATK.tar.gz was not transmitted completly

    Thanks for your support

  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @bshifaw Thank you for bringing this to my attention. @SHollizeck Could you try the 4.1.4.0 release: https://github.com/broadinstitute/gatk/releases/tag/4.1.4.0 and let me know how it goes? If it still fails I will try to debug promptly.

  • SHollizeckSHollizeck Member

    @davidben said:
    @bshifaw Thank you for bringing this to my attention. @SHollizeck Could you try the 4.1.4.0 release: https://github.com/broadinstitute/gatk/releases/tag/4.1.4.0 and let me know how it goes? If it still fails I will try to debug promptly.

    Do you want me to rerun the whole workflow? because that would take about 10 days, or just the filtering?

  • SHollizeckSHollizeck Member

    I reran just the FilterMutectCalls part with the newest Docker image and it failed with the same error message at the same time

    06:53:07.851 INFO  FilterMutectCalls - ------------------------------------------------------------
    06:53:07.851 INFO  FilterMutectCalls - The Genome Analysis Toolkit (GATK) v4.1.4.0
    06:53:07.851 INFO  FilterMutectCalls - For support and documentation go to https://software.broadinstitute.org/gatk/
    06:53:07.851 INFO  FilterMutectCalls - Executing as [email protected] on Linux v3.10.0-957.21.3.el7.x86_64 amd64
    06:53:07.852 INFO  FilterMutectCalls - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_212-8u212-b03-0ubuntu1.16.04.1-b03
    06:53:07.852 INFO  FilterMutectCalls - Start Date/Time: October 14, 2019 6:53:06 AM UTC
    06:53:07.852 INFO  FilterMutectCalls - ------------------------------------------------------------
    06:53:07.852 INFO  FilterMutectCalls - ------------------------------------------------------------
    06:53:07.852 INFO  FilterMutectCalls - HTSJDK Version: 2.20.3
    06:53:07.852 INFO  FilterMutectCalls - Picard Version: 2.21.1
    06:53:07.852 INFO  FilterMutectCalls - HTSJDK Defaults.COMPRESSION_LEVEL : 2
    06:53:07.853 INFO  FilterMutectCalls - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    06:53:07.853 INFO  FilterMutectCalls - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    06:53:07.853 INFO  FilterMutectCalls - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    06:53:07.853 INFO  FilterMutectCalls - Deflater: IntelDeflater
    06:53:07.853 INFO  FilterMutectCalls - Inflater: IntelInflater
    06:53:07.853 INFO  FilterMutectCalls - GCS max retries/reopens: 20
    06:53:07.853 INFO  FilterMutectCalls - Requester pays: disabled
    06:53:07.853 INFO  FilterMutectCalls - Initializing engine
    06:53:08.338 INFO  FeatureManager - Using codec VCFCodec to read file file:///researchers/sebastian.hollizeck/./cromwell-execution/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/164276910/generated-58e6fad0-db7e-11e9-9da2-005056b01e3e.vcf.gz
    06:53:08.580 INFO  FilterMutectCalls - Done initializing engine
    06:53:09.216 INFO  IOUtils - Extracting data from archive: file:///researchers/sebastian.hollizeck/./cromwell-execution/sebastian/f3c8dc32-7754-42c3-b0a7-9d667904c2e5/call-filtering/inputs/-1768654832/generated-58e75278-db7e-11e9-9da2-005056b01e3e.tar.gz
    06:53:09.231 INFO  IOUtils - Extracting file: ./b_mutated.orientation_priors
    06:53:09.232 INFO  IOUtils - Extracting file: ./g_mutated.orientation_priors
    06:53:09.233 INFO  IOUtils - Extracting file: ./c_mutated.orientation_priors
    06:53:09.233 INFO  IOUtils - Extracting file: ./f_mutated.orientation_priors
    06:53:09.234 INFO  IOUtils - Extracting file: ./i_mutated.orientation_priors
    06:53:09.234 INFO  IOUtils - Extracting file: ./d_mutated.orientation_priors
    06:53:09.234 INFO  IOUtils - Extracting file: ./e_mutated.orientation_priors
    06:53:09.235 INFO  IOUtils - Extracting file: ./a_mutated.orientation_priors
    06:53:09.235 INFO  IOUtils - Extracting file: ./h_mutated.orientation_priors
    06:53:09.236 INFO  IOUtils - Extracting file: ./j_mutated.orientation_priors
    06:53:09.307 INFO  ProgressMeter - Starting traversal
    06:53:09.307 INFO  ProgressMeter -        Current Locus  Elapsed Minutes    Variants Processed  Variants/Minute
    06:53:09.308 INFO  FilterMutectCalls - Starting pass 0 through the variants
    06:53:19.376 INFO  ProgressMeter -            1:1839944              0.2                 14000          83432.7
    .
    .
    .
    12:22:34.335 INFO  FilterMutectCalls - Finished pass 0 through the variants
    12:36:03.467 INFO  FilterMutectCalls - Shutting down engine
    [October 14, 2019 12:36:03 PM UTC] org.broadinstitute.hellbender.tools.walkers.mutect.filtering.FilterMutectCalls done. Elapsed time: 342.95 minutes.
    Runtime.totalMemory()=20146814976
    java.lang.IllegalArgumentException: beta must be greater than 0 but got -87566.7500301585
        at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:725)
        at org.broadinstitute.hellbender.tools.walkers.readorientation.BetaDistributionShape.<init>(BetaDistributionShape.java:14)
        at org.broadinstitute.hellbender.tools.walkers.mutect.clustering.BinomialCluster.getFuzzyBinomial(BinomialCluster.java:42)
        at org.broadinstitute.hellbender.tools.walkers.mutect.clustering.BinomialCluster.learn(BinomialCluster.java:33)
        at org.broadinstitute.hellbender.tools.walkers.mutect.clustering.SomaticClusteringModel.lambda$learnAndClearAccumulatedData$7(SomaticClusteringModel.java:131)
        at org.broadinstitute.hellbender.utils.IndexRange.forEach(IndexRange.java:116)
        at org.broadinstitute.hellbender.tools.walkers.mutect.clustering.SomaticClusteringModel.learnAndClearAccumulatedData(SomaticClusteringModel.java:131)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.Mutect2FilteringEngine.learnParameters(Mutect2FilteringEngine.java:156)
        at org.broadinstitute.hellbender.tools.walkers.mutect.filtering.FilterMutectCalls.afterNthPass(FilterMutectCalls.java:157)
        at org.broadinstitute.hellbender.engine.MultiplePassVariantWalker.traverse(MultiplePassVariantWalker.java:44)
        at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1048)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206)
        at org.broadinstitute.hellbender.Main.main(Main.java:292)
    
    
  • davidbendavidben BostonMember, Broadie, Dev ✭✭✭

    @SHollizeck Are you able to share the unfiltered VCF?

  • SHollizeckSHollizeck Member

    I just checked it is in the ftp upload that I submitted.

Sign In or Register to comment.