Heads up:
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.
Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!
Test-drive the GATK tools and Best Practices pipelines on Terra
Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office for a Broad Institute event from Dec 10th to Dec 11th 2019. We will be back to monitor the GATK forum on Dec 12th 2019. In the meantime we encourage you to help out other community members with their queries.
Thank you for your patience!
We will be out of the office for a Broad Institute event from Dec 10th to Dec 11th 2019. We will be back to monitor the GATK forum on Dec 12th 2019. In the meantime we encourage you to help out other community members with their queries.
Thank you for your patience!
VariantRecalibrator: java.nio.file.NoSuchFileException

GATK 4.1.1.0, bash, linux server
Hi,
I have the following error/ exception... maybe is always related to this [thread]!?(https://gatkforums.broadinstitute.org/gatk/discussion/10797/picard-revertsam-java-nio-file-nosuchfileexception#latest "thread")!?
Best
Running: java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx20g -XX:GCTimeLimit=50 -XX:GCHeapFreeLimit=10 -XX:ConcGCThreads=1 -XX:ParallelGCThreads=2 -jar /share/apps/bio/gatk-4.1.1.0/gatk-package-4.1.1.0-local.jar VariantRecalibrator -V 3.rawHFSO.vcf.gz -O 4i.rawHFSO-iVR.vcf.gz --tranches-file 4i.indel.tranches --trust-all-polymorphic -tranche 100.0 -tranche 99.95 -tranche 99.9 -tranche 99.5 -tranche 99.0 -tranche 97.0 -tranche 96.0 -tranche 95.0 -tranche 94.0 -tranche 93.5 -tranche 93.0 -tranche 92.0 -tranche 91.0 -tranche 90.0 -an FS -an ReadPosRankSum -an MQRankSum -an QD -an SOR -mode INDEL --max-gaussians 4 -resource:mills,known=false,training=true,truth=true,prior=12 /shared/resources/gatk4hg38db/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz -resource:axiomPoly,known=false,training=true,truth=false,prior=10 /shared/resources/gatk4hg38db/Axiom_Exome_Plus.genotypes.all_populations.poly.hg38.vcf.gz -resource:dbsnp,known=true,training=false,truth=false,prior=2 /shared/resources/gatk4hg38db/Homo_sapiens_assembly38.dbsnp138.vcf --rscript-file indel.plots.R --tmp-dir /home/manolis/GATK4/tmp/ 06:29:53.083 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/share/apps/bio/gatk-4.1.1.0/gatk-package-4.1.1.0-local.jar!/com/intel/gkl/native/libgkl_compression.so Exception in thread "Thread-1" htsjdk.samtools.util.RuntimeIOException: java.nio.file.NoSuchFileException: /home/manolis/GATK4/tmp/Rlib.667992084951395367 at htsjdk.samtools.util.IOUtil.recursiveDelete(IOUtil.java:1346) at org.broadinstitute.hellbender.utils.io.IOUtils.deleteRecursively(IOUtils.java:1061) at org.broadinstitute.hellbender.utils.io.DeleteRecursivelyOnExitPathHook.runHooks(DeleteRecursivelyOnExitPathHook.java:56) at java.lang.Thread.run(Thread.java:745) Caused by: java.nio.file.NoSuchFileException: /home/manolis/GATK4/tmp/Rlib.667992084951395367 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) at java.nio.file.Files.readAttributes(Files.java:1737) at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219) at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276) at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322) at java.nio.file.Files.walkFileTree(Files.java:2662) at java.nio.file.Files.walkFileTree(Files.java:2742) at htsjdk.samtools.util.IOUtil.recursiveDelete(IOUtil.java:1344) ... 3 more Running: java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx100g -XX:GCTimeLimit=50 -XX:GCHeapFreeLimit=10 -XX:ConcGCThreads=1 -XX:ParallelGCThreads=2 -jar /share/apps/bio/gatk-4.1.1.0/gatk-package-4.1.1.0-local.jar VariantRecalibrator -V 3.rawHFSO.vcf.gz -O 4s.rawHFSO-sVR.vcf.gz --tranches-file 4s.snp.tranches -trust-all-polymorphic -tranche 100.0 -tranche 99.95 -tranche 99.9 -tranche 99.8 -tranche 99.6 -tranche 99.5 -tranche 99.4 -tranche 99.3 -tranche 99.0 -tranche 98.0 -tranche 97.0 -tranche 90.0 -an QD -an MQRankSum -an ReadPosRankSum -an FS -an MQ -an SOR -mode SNP --max-gaussians 6 -resource:hapmap,known=false,training=true,truth=true,prior=15 /shared/resources/gatk4hg38db/hapmap_3.3.hg38.vcf.gz -resource:omni,known=false,training=true,truth=true,prior=12 /shared/resources/gatk4hg38db/1000G_omni2.5.hg38.vcf.gz -resource:1000G,known=false,training=true,truth=false,prior=10 /shared/resources/gatk4hg38db/1000G_phase1.snps.high_confidence.hg38.vcf.gz -resource:dbsnp,known=true,training=false,truth=false,prior=7 /shared/resources/gatk4hg38db/Homo_sapiens_assembly38.dbsnp138.vcf --rscript-file snp.plots.R --tmp-dir /home/manolis/GATK4/tmp/ Exception in thread "Thread-1" htsjdk.samtools.util.RuntimeIOException: java.nio.file.NoSuchFileException: /home/manolis/GATK4/tmp/Rlib.9072785099028549732 at htsjdk.samtools.util.IOUtil.recursiveDelete(IOUtil.java:1346) at org.broadinstitute.hellbender.utils.io.IOUtils.deleteRecursively(IOUtils.java:1061) at org.broadinstitute.hellbender.utils.io.DeleteRecursivelyOnExitPathHook.runHooks(DeleteRecursivelyOnExitPathHook.java:56) at java.lang.Thread.run(Thread.java:745) Caused by: java.nio.file.NoSuchFileException: /home/manolis/GATK4/tmp/Rlib.9072785099028549732 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) at java.nio.file.Files.readAttributes(Files.java:1737) at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219) at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276) at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322) at java.nio.file.Files.walkFileTree(Files.java:2662) at java.nio.file.Files.walkFileTree(Files.java:2742) at htsjdk.samtools.util.IOUtil.recursiveDelete(IOUtil.java:1344) ... 3 more
Tagged:
Best Answer
-
cnorman United States ✭✭
This issue has been fixed in github, but GATK hasn't been released since the fix went in. So 4.1.2.0 still has this behavior. If someone does see it in a recent nightly build we can revisit, but I haven't seen anyone report that.
I expect that when you see this exception its mostly harmless, though annoying, since its just reporting that it can't delete a temp file that has already been deleted. Hope that helps.
Answers
I get the same issue. I've also tried running without the
--tmp-dir
option, and the same stack trace is produced. I'm wondering if in this case it would be fine to keep going, or if there's another problem lurking before this exception.In my case it runs through, and seems to run Rscript for generating the plots, and then crashes.
Update: Found this comment here (in the link provided in the OP question), which points to a likely cause. https://gatkforums.broadinstitute.org/gatk/discussion/comment/58104/#Comment_58104 .
The issue being tracked is https://github.com/broadinstitute/gatk/issues/5893 and seems to match the symptoms. Got this error on 4.1.1.0 as well. Will try again on a more recent build and report.
Retrying with broadinstitute/gatk:4.1.2.0 produced the same error outcome. (The patch for the bug was applied 16 days ago, and 4.1.2.0 was released 22 days ago, so not a huge surprise). Will try the gatk-nightly build. (tag 2019-05-15-4.1.2.0-7-ga229b5646-NIGHTLY-SNAPSHOT)
If the said patch is truly a fix for this, the error would be with RScriptExecutor. As a workaround, it might be possible to simply remove
Rscript
from $PATH just before calling VariantRecalibrator, and then following that with the equivalent Rscript commands:where
<recal.Rscript>
is the argument to--rscript-file
, and<tranches>
is the argument to--tranches-file
.the plot_Tranches.R is deep in the jar file, but it can be extracted with:
This may be the same issue. I opened a new issue in case it is not:
You can track the progress on this issue here
This issue has been fixed in github, but GATK hasn't been released since the fix went in. So 4.1.2.0 still has this behavior. If someone does see it in a recent nightly build we can revisit, but I haven't seen anyone report that.
I expect that when you see this exception its mostly harmless, though annoying, since its just reporting that it can't delete a temp file that has already been deleted. Hope that helps.
Thank you @AdelaideR and @cnorman!
Best