Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office on November 11th and 13th 2019, due to the U.S. holiday(Veteran's day) and due to a team event(Nov 13th). We will return to monitoring the GATK forum on November 12th and 14th respectively. Thank you for your patience.

PlotDenoisedCopyRatios error at generating plots

Hi

I am trying to generate some plots using PlotDenoisedCopyRatios function in GATK4.1.2.0.
Everything seems to work nicely until the R script is initialized. I got the and error regarding --sample_name:

gatk PlotDenoisedCopyRatios --standardized-copy-ratios try_sta --denoised-copy-ratios try --sequence-dictionary /bicoh/MARGenomics/Ref_Genomes_fa/GATK_bundle/hg38/ref/Homo_sapiens_assembly38.dict --minimum-contig-length 46709983 --output Plots --output-prefix Sample_7387
Using GATK jar /soft/EB_repo/bio/sequence/programs/noarch/GATK/4.1.2.0/gatk-package-4.1.2.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /soft/EB_repo/bio/sequence/programs/noarch/GATK/4.1.2.0/gatk-package-4.1.2.0-local.jar PlotDenoisedCopyRatios --standardized-copy-ratios try_sta --denoised-copy-ratios try --sequence-dictionary /bicoh/MARGenomics/Ref_Genomes_fa/GATK_bundle/hg38/ref/Homo_sapiens_assembly38.dict --minimum-contig-length 46709983 --output Plots --output-prefix Sample_7387
13:26:45.991 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/soft/EB_repo/bio/sequence/programs/noarch/GATK/4.1.2.0/gatk-package-4.1.2.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
Oct 31, 2019 1:28:08 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
13:28:08.914 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
13:28:08.916 INFO  PlotDenoisedCopyRatios - The Genome Analysis Toolkit (GATK) v4.1.2.0
13:28:08.916 INFO  PlotDenoisedCopyRatios - For support and documentation go to https://software.broadinstitute.org/gatk/
13:28:08.917 INFO  PlotDenoisedCopyRatios - Executing as [email protected] on Linux v3.10.0-862.14.4.el7.x86_64 amd64
13:28:08.918 INFO  PlotDenoisedCopyRatios - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_92-b14
13:28:08.918 INFO  PlotDenoisedCopyRatios - Start Date/Time: October 31, 2019 1:26:45 PM CET
13:28:08.918 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
13:28:08.918 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
13:28:08.927 INFO  PlotDenoisedCopyRatios - HTSJDK Version: 2.19.0
13:28:08.928 INFO  PlotDenoisedCopyRatios - Picard Version: 2.19.0
13:28:08.928 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.COMPRESSION_LEVEL : 2
13:28:08.928 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
13:28:08.928 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
13:28:08.928 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
13:28:08.928 INFO  PlotDenoisedCopyRatios - Deflater: IntelDeflater
13:28:08.928 INFO  PlotDenoisedCopyRatios - Inflater: IntelInflater
13:28:08.928 INFO  PlotDenoisedCopyRatios - GCS max retries/reopens: 20
13:28:08.929 INFO  PlotDenoisedCopyRatios - Requester pays: disabled
13:28:08.929 INFO  PlotDenoisedCopyRatios - Initializing engine
13:28:08.929 INFO  PlotDenoisedCopyRatios - Done initializing engine
13:28:08.943 INFO  PlotDenoisedCopyRatios - Reading and validating input files...
13:28:14.567 INFO  PlotDenoisedCopyRatios - Contigs above length threshold: {chr1=248956422, chr2=242193529, chr3=198295559, chr4=190214555, chr5=181538259, chr6=170805979, chr7=159345973, chr8=145138636, chr9=138394717, chr10=133797422, chr11=135086622, chr12=133275309, chr13=114364328, chr14=107043718, chr15=101991189, chr16=90338345, chr17=83257441, chr18=80373285, chr19=58617616, chr20=64444167, chr21=46709983, chr22=50818468, chrX=156040895, chrY=57227415}
13:28:14.638 WARN  PlotDenoisedCopyRatios - Contigs present in the file try_sta are missing from the sequence dictionary and will not be plotted.
13:28:14.797 WARN  PlotDenoisedCopyRatios - Contigs present in the file try are missing from the sequence dictionary and will not be plotted.
13:28:14.873 INFO  PlotDenoisedCopyRatios - Writing plots to /users/genomics/jgibert/Exomes_Bea_uBAM/CNV_somatic/Plots...
13:28:28.947 INFO  PlotDenoisedCopyRatios - Shutting down engine
[October 31, 2019 1:28:28 PM CET] org.broadinstitute.hellbender.tools.copynumber.plotting.PlotDenoisedCopyRatios done. Elapsed time: 1.72 minutes.
Runtime.totalMemory()=737148928
org.broadinstitute.hellbender.utils.R.RScriptExecutorException: 
Rscript exited with 1
Command Line: Rscript -e tempLibDir = '/tmp/Rlib.1345838355110254415';source('/tmp/CNVPlottingLibrary.6888862715841197009.R');source('/tmp/PlotDenoisedCopyRatios.3695016065488923741.R'); --args --sample_name=-A --standardized_copy_ratios_file=/users/genomics/jgibert/Exomes_Bea_uBAM/CNV_somatic/try_sta --denoised_copy_ratios_file=/users/genomics/jgibert/Exomes_Bea_uBAM/CNV_somatic/try --contig_names=chr1CONTIG_DELIMITERchr2CONTIG_DELIMITERchr3CONTIG_DELIMITERchr4CONTIG_DELIMITERchr5CONTIG_DELIMITERchr6CONTIG_DELIMITERchr7CONTIG_DELIMITERchr8CONTIG_DELIMITERchr9CONTIG_DELIMITERchr10CONTIG_DELIMITERchr11CONTIG_DELIMITERchr12CONTIG_DELIMITERchr13CONTIG_DELIMITERchr14CONTIG_DELIMITERchr15CONTIG_DELIMITERchr16CONTIG_DELIMITERchr17CONTIG_DELIMITERchr18CONTIG_DELIMITERchr19CONTIG_DELIMITERchr20CONTIG_DELIMITERchr21CONTIG_DELIMITERchr22CONTIG_DELIMITERchrXCONTIG_DELIMITERchrY --contig_lengths=248956422CONTIG_DELIMITER242193529CONTIG_DELIMITER198295559CONTIG_DELIMITER190214555CONTIG_DELIMITER181538259CONTIG_DELIMITER170805979CONTIG_DELIMITER159345973CONTIG_DELIMITER145138636CONTIG_DELIMITER138394717CONTIG_DELIMITER133797422CONTIG_DELIMITER135086622CONTIG_DELIMITER133275309CONTIG_DELIMITER114364328CONTIG_DELIMITER107043718CONTIG_DELIMITER101991189CONTIG_DELIMITER90338345CONTIG_DELIMITER83257441CONTIG_DELIMITER80373285CONTIG_DELIMITER58617616CONTIG_DELIMITER64444167CONTIG_DELIMITER46709983CONTIG_DELIMITER50818468CONTIG_DELIMITER156040895CONTIG_DELIMITER57227415 --output_dir=/users/genomics/jgibert/Exomes_Bea_uBAM/CNV_somatic/Plots/ --output_prefix=Sample_7387
Stdout: 
Stderr: Error in make_option(c("--sample_name", "-sample_name"), dest = "sample_name",  : 
  Short flag -sample_name must only be a '-' and a single letter
Calls: source -> withVisible -> eval -> eval -> make_option

    at org.broadinstitute.hellbender.utils.R.RScriptExecutor.getScriptException(RScriptExecutor.java:80)
    at org.broadinstitute.hellbender.utils.R.RScriptExecutor.getScriptException(RScriptExecutor.java:19)
    at org.broadinstitute.hellbender.utils.runtime.ScriptExecutor.executeCuratedArgs(ScriptExecutor.java:126)
    at org.broadinstitute.hellbender.utils.R.RScriptExecutor.exec(RScriptExecutor.java:129)
    at org.broadinstitute.hellbender.tools.copynumber.plotting.PlotDenoisedCopyRatios.writeDenoisingPlots(PlotDenoisedCopyRatios.java:204)
    at org.broadinstitute.hellbender.tools.copynumber.plotting.PlotDenoisedCopyRatios.doWork(PlotDenoisedCopyRatios.java:155)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
    at org.broadinstitute.hellbender.Main.main(Main.java:291)
Exception in thread "Thread-1" htsjdk.samtools.util.RuntimeIOException: java.nio.file.NoSuchFileException: /tmp/Rlib.1345838355110254415
    at htsjdk.samtools.util.IOUtil.recursiveDelete(IOUtil.java:1346)
    at org.broadinstitute.hellbender.utils.io.IOUtils.deleteRecursively(IOUtils.java:1061)
    at org.broadinstitute.hellbender.utils.io.DeleteRecursivelyOnExitPathHook.runHooks(DeleteRecursivelyOnExitPathHook.java:56)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.file.NoSuchFileException: /tmp/Rlib.1345838355110254415
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
    at java.nio.file.Files.readAttributes(Files.java:1737)
    at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
    at java.nio.file.Files.walkFileTree(Files.java:2662)
    at java.nio.file.Files.walkFileTree(Files.java:2742)
    at htsjdk.samtools.util.IOUtil.recursiveDelete(IOUtil.java:1344)
    ... 3 more

I tried to follow the error and I changed the @RG SM: field in both denoised and standarized tsv files with the requirements (as you may see in the log), but the error still persists.

Any idea why this is happening?
Thanks!

Tagged:

Best Answer

Answers

  • JoanGibertJoanGibert Member

    So, updating to GATK 4.1.4.0 should solve both issues?

  • sleeslee Member, Broadie, Dev ✭✭✭

    Unfortunately, the fix for the first issue was merged to master after the 4.1.4.0 release. (But I'd go ahead and update anyway, unless you have a reason not to!)

  • jejacobs23jejacobs23 Portland, ORMember

    Hi @slee, I am experiencing the same issue as described by @JoanGibert when trying to run PlotDenosedCopyRatios on GATK 4.1.3.0. Here is my stderr file:

    Using GATK jar /home/exacloud/lustre1/jjacobs/programs/gatk-4.1.3.0/gatk-package-4.1.3.0-local.jar
    Running:
        java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/exacloud/lustre1/jjacobs/programs/gatk-4.1.3.0/gatk-package-4$
    00:52:23.708 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/exacloud/lustre1/jjacobs/programs/gatk-4.1.3.0/gatk-package-4.1.3.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
    Nov 02, 2019 12:52:25 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
    INFO: Failed to detect whether we are running on Google Compute Engine.
    00:52:25.793 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
    00:52:25.794 INFO  PlotDenoisedCopyRatios - The Genome Analysis Toolkit (GATK) v4.1.3.0
    00:52:25.794 INFO  PlotDenoisedCopyRatios - For support and documentation go to https://software.broadinstitute.org/gatk/
    00:52:25.795 INFO  PlotDenoisedCopyRatios - Executing as [email protected] on Linux v3.10.0-862.14.4.el7.x86_64 amd64
    00:52:25.795 INFO  PlotDenoisedCopyRatios - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_191-b12
    00:52:25.796 INFO  PlotDenoisedCopyRatios - Start Date/Time: November 2, 2019 12:52:23 AM PDT
    00:52:25.796 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
    00:52:25.796 INFO  PlotDenoisedCopyRatios - ------------------------------------------------------------
    00:52:25.797 INFO  PlotDenoisedCopyRatios - HTSJDK Version: 2.20.1
    00:52:25.797 INFO  PlotDenoisedCopyRatios - Picard Version: 2.20.5
    00:52:25.797 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.COMPRESSION_LEVEL : 2
    00:52:25.797 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    00:52:25.797 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    00:52:25.798 INFO  PlotDenoisedCopyRatios - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    00:52:25.798 INFO  PlotDenoisedCopyRatios - Deflater: IntelDeflater
    00:52:25.798 INFO  PlotDenoisedCopyRatios - Inflater: IntelInflater
    00:52:25.798 INFO  PlotDenoisedCopyRatios - GCS max retries/reopens: 20
    00:52:25.798 INFO  PlotDenoisedCopyRatios - Requester pays: disabled
    00:52:25.799 INFO  PlotDenoisedCopyRatios - Initializing engine
    00:52:25.799 INFO  PlotDenoisedCopyRatios - Done initializing engine
    00:52:25.800 INFO  PlotDenoisedCopyRatios - Reading and validating input files...
    00:52:38.171 INFO  PlotDenoisedCopyRatios - Contigs above length threshold: {chr1=248956422, chr2=242193529, chr3=198295559, chr4=190214555, chr5=181538259, chr6=170805979, chr7=159345973, chr8=145138636, chr9=138394717, chr10=133797422$
    00:52:38.504 WARN  PlotDenoisedCopyRatios - Contigs present in the file /home/exacloud/lustre1/jjacobs/data/osteo/SJOS001101_M1/T_clean.standardizedCR.tsv are missing from the sequence dictionary and will not be plotted.
    00:52:39.372 WARN  PlotDenoisedCopyRatios - Contigs present in the file /home/exacloud/lustre1/jjacobs/data/osteo/SJOS001101_M1/T_clean.denoisedCR.tsv are missing from the sequence dictionary and will not be plotted.
    00:52:39.593 INFO  PlotDenoisedCopyRatios - Writing plots to /home/exacloud/lustre1/jjacobs/data/osteo/SJOS001101_M1/plots...
    00:52:40.141 INFO  PlotDenoisedCopyRatios - Shutting down engine
    [November 2, 2019 12:52:40 AM PDT] org.broadinstitute.hellbender.tools.copynumber.plotting.PlotDenoisedCopyRatios done. Elapsed time: 0.27 minutes.
    Runtime.totalMemory()=2039545856
    org.broadinstitute.hellbender.utils.R.RScriptExecutorException:
    Rscript exited with 1
    Command Line: Rscript -e tempLibDir = '/tmp/Rlib.1122299453047432152';source('/tmp/CNVPlottingLibrary.236356716369574865.R');source('/tmp/PlotDenoisedCopyRatios.5523146249630074228.R'); --args --sample_name=H_LC-SJOS001101-M1 --standard$
    Stdout:
    Stderr: Error in make_option(c("--sample_name", "-sample_name"), dest = "sample_name",  :
      Short flag -sample_name must only be a '-' and a single letter
    Calls: source -> withVisible -> eval -> eval -> make_option
    
            at org.broadinstitute.hellbender.utils.R.RScriptExecutor.getScriptException(RScriptExecutor.java:80)
            at org.broadinstitute.hellbender.utils.R.RScriptExecutor.getScriptException(RScriptExecutor.java:19)
            at org.broadinstitute.hellbender.utils.runtime.ScriptExecutor.executeCuratedArgs(ScriptExecutor.java:126)
            at org.broadinstitute.hellbender.utils.R.RScriptExecutor.exec(RScriptExecutor.java:126)
            at org.broadinstitute.hellbender.tools.copynumber.plotting.PlotDenoisedCopyRatios.writeDenoisingPlots(PlotDenoisedCopyRatios.java:204)
            at org.broadinstitute.hellbender.tools.copynumber.plotting.PlotDenoisedCopyRatios.doWork(PlotDenoisedCopyRatios.java:155)
            at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
            at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
            at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
            at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
            at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
            at org.broadinstitute.hellbender.Main.main(Main.java:291)
    srun: error: exanode-3-2: task 0: Exited with exit code 3
    

    Can I ask a clarifying question? What is meant by "build from the latest master"? Does that mean reinstalling GATK?

  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin

    @jejacobs23

    See the workaround provided for this issue above.

  • sleeslee Member, Broadie, Dev ✭✭✭

    @jejacobs23 To "build from the latest master" means to check out the master branch from GitHub and build a jar from it (see https://github.com/broadinstitute/gatk/blob/master/README.md#building). The master branch may include changes that have not yet been officially released.

Sign In or Register to comment.