# in file names converted to %23 resulting in file not found

TechnicalVaultTechnicalVault Cambridge, UKMember
edited January 31 in Ask the GATK team

Whilst I was trying to run CombineGVCFs 4.0.0 I got a very strange error, file not found for a file I knew existed. Looking into the backtrace it looks like somehow a # is getting mistakenly URL escaped?

org.broadinstitute.hellbender.exceptions.GATKException: Error initializing feature reader for path /project/gvcf-pcr/23232_1#1/1.g.vcf.gz
    at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:341)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.getFeatureReader(FeatureDataSource.java:292)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.<init>(FeatureDataSource.java:244)
    at org.broadinstitute.hellbender.engine.FeatureManager.addToFeatureSources(FeatureManager.java:202)
    at org.broadinstitute.hellbender.engine.MultiVariantWalker.lambda$initializeDrivingVariants$0(MultiVariantWalker.java:66)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
    at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
    at org.broadinstitute.hellbender.engine.MultiVariantWalker.initializeDrivingVariants(MultiVariantWalker.java:56)
    at org.broadinstitute.hellbender.engine.VariantWalkerBase.initializeFeatures(VariantWalkerBase.java:47)
    at org.broadinstitute.hellbender.engine.GATKTool.onStartup(GATKTool.java:558)
    at org.broadinstitute.hellbender.engine.MultiVariantWalker.onStartup(MultiVariantWalker.java:48)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:152)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:195)
    at org.broadinstitute.hellbender.Main.main(Main.java:275)
Caused by: htsjdk.tribble.TribbleException$MalformedFeatureFile: Unable to create BasicFeatureReader using feature file , for input source: file:///project/gvcf-pcr/23232_1%231/1.g.vcf.gz
    at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:113)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:337)
    ... 16 more
Caused by: java.io.FileNotFoundException: /project/gvcf-pcr/23232_1%231/1.g.vcf.gz (No such file or directory)
    at java.io.RandomAccessFile.open0(Native Method)
    at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
    at htsjdk.samtools.seekablestream.SeekableFileStream.<init>(SeekableFileStream.java:47)
    at htsjdk.samtools.seekablestream.SeekableStreamFactory$DefaultSeekableStreamFactory.getStreamFor(SeekableStreamFactory.java:99)
    at htsjdk.tribble.readers.TabixReader.<init>(TabixReader.java:129)
    at htsjdk.tribble.TabixFeatureReader.<init>(TabixFeatureReader.java:83)
    at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:106)
    ... 17 more

Issue · Github
by Sheila

Issue Number
2894
State
closed
Last Updated
Assignee
Array
Closed By
chandrans

Best Answer

Answers

  • TechnicalVaultTechnicalVault Cambridge, UKMember

    Replicated in GATK 4.0.1.0 :(

  • shleeshlee CambridgeMember, Broadie, Moderator

    Please post your exact command that produced the error @TechnicalVault.

  • TechnicalVaultTechnicalVault Cambridge, UKMember
    edited February 2

    The command line and full run log was as follows, the tmp file was just a list of paths to the .g.vcf.gz files.

    The file in question was listed as
    /lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1#1/1.g.vcf.gz

    Using GATK jar /lustre/scratch115/realdata/mdt2/projects/gdap-wgs/gvcf-4.0/scripts/gatk-4.0.1.0/gatk-package-4.0.1.0-local.jar
    Running:
        /software/jre1.8.0_74/bin/java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -Djava.io.tmpdir=/lustre/scratch115/projects/gdap-wgs/gvcf-4.0/tmp -XX:-UsePerfData -Xrs -Xmx3200m -jar /lustre/scratch115/realdata/mdt2/projects/gdap-wgs/gvcf-4.0/scripts/gatk-4.0.1.0/gatk-package-4.0.1.0-local.jar CombineGVCFs -R /lustre/scratch115/resources/ref/Homo_sapiens/HS38DH/hs38DH.fa -V /tmp/tmp.7dp5SuO2TD.list -O /lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr_combined/1_1.g.vcf.gz -L /lustre/scratch115/resources/ref/Homo_sapiens/HS38DH/intervals/arvados/wgs_calling_regions.hg38.interval_list.1_of_200.interval_list
    12:12:12.775 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/lustre/scratch115/realdata/mdt2/projects/gdap-wgs/gvcf-4.0/scripts/gatk-4.0.1.0/gatk-package-4.0.1.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
    12:12:13.728 INFO  CombineGVCFs - ------------------------------------------------------------
    12:12:13.728 INFO  CombineGVCFs - The Genome Analysis Toolkit (GATK) v4.0.1.0
    12:12:13.729 INFO  CombineGVCFs - For support and documentation go to https://software.broadinstitute.org/gatk/
    12:12:13.729 INFO  CombineGVCFs - Executing as mp15@bc-25-1-04 on Linux v3.2.0-105-generic amd64
    12:12:13.730 INFO  CombineGVCFs - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_74-b02
    12:12:13.730 INFO  CombineGVCFs - Start Date/Time: 02 February 2018 12:12:12 GMT
    12:12:13.731 INFO  CombineGVCFs - ------------------------------------------------------------
    12:12:13.731 INFO  CombineGVCFs - ------------------------------------------------------------
    12:12:13.732 INFO  CombineGVCFs - HTSJDK Version: 2.14.1
    12:12:13.732 INFO  CombineGVCFs - Picard Version: 2.17.2
    12:12:13.732 INFO  CombineGVCFs - HTSJDK Defaults.COMPRESSION_LEVEL : 1
    12:12:13.733 INFO  CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    12:12:13.733 INFO  CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    12:12:13.733 INFO  CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    12:12:13.733 INFO  CombineGVCFs - Deflater: IntelDeflater
    12:12:13.733 INFO  CombineGVCFs - Inflater: IntelInflater
    12:12:13.734 INFO  CombineGVCFs - GCS max retries/reopens: 20
    12:12:13.734 INFO  CombineGVCFs - Using google-cloud-java patch 6d11bef1c81f885c26b2b56c8616b7a705171e4f from https://github.com/droazen/google-cloud-java/tree/dr_all_nio_fixes
    12:12:13.734 INFO  CombineGVCFs - Initializing engine
    12:12:16.228 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13939_6/1.g.vcf.gz
    12:12:16.453 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13939_7/1.g.vcf.gz
    12:12:16.582 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13939_8/1.g.vcf.gz
    12:12:16.735 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13940_5/1.g.vcf.gz
    12:12:16.833 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13940_6/1.g.vcf.gz
    12:12:16.975 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13940_7/1.g.vcf.gz
    12:12:17.124 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13940_8/1.g.vcf.gz
    12:12:17.229 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13964_1/1.g.vcf.gz
    12:12:17.336 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/14538_6/1.g.vcf.gz
    12:12:17.427 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16006_4/1.g.vcf.gz
    12:12:17.558 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16006_5/1.g.vcf.gz
    12:12:17.651 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16006_6/1.g.vcf.gz
    12:12:17.756 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_1/1.g.vcf.gz
    12:12:17.878 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_2/1.g.vcf.gz
    12:12:17.985 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_3/1.g.vcf.gz
    12:12:18.086 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_4/1.g.vcf.gz
    12:12:18.180 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_5/1.g.vcf.gz
    12:12:18.319 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_6/1.g.vcf.gz
    12:12:18.444 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_7/1.g.vcf.gz
    12:12:18.565 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16014_4/1.g.vcf.gz
    12:12:18.675 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16014_6/1.g.vcf.gz
    12:12:18.771 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16014_7/1.g.vcf.gz
    12:12:18.892 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16015_1/1.g.vcf.gz
    12:12:18.996 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16015_3/1.g.vcf.gz
    12:12:19.116 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16015_4/1.g.vcf.gz
    12:12:19.232 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16015_7/1.g.vcf.gz
    12:12:19.399 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_4/1.g.vcf.gz
    12:12:19.487 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_5/1.g.vcf.gz
    12:12:19.572 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_6/1.g.vcf.gz
    12:12:19.762 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_7/1.g.vcf.gz
    12:12:19.856 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_8/1.g.vcf.gz
    12:12:19.961 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16063_3/1.g.vcf.gz
    12:12:20.051 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16063_4/1.g.vcf.gz
    12:12:20.130 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16063_6/1.g.vcf.gz
    12:12:20.248 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16063_7/1.g.vcf.gz
    12:12:20.327 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16279_1/1.g.vcf.gz
    12:12:20.420 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16279_2/1.g.vcf.gz
    12:12:20.554 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16279_3/1.g.vcf.gz
    12:12:20.652 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_1/1.g.vcf.gz
    12:12:20.739 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_2/1.g.vcf.gz
    12:12:20.823 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_3/1.g.vcf.gz
    12:12:20.920 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_4/1.g.vcf.gz
    12:12:21.013 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_5/1.g.vcf.gz
    12:12:21.114 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_6/1.g.vcf.gz
    12:12:21.199 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_7/1.g.vcf.gz
    12:12:21.270 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_8/1.g.vcf.gz
    12:12:21.351 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_1/1.g.vcf.gz
    12:12:21.438 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_2/1.g.vcf.gz
    12:12:21.544 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_3/1.g.vcf.gz
    12:12:21.611 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_4/1.g.vcf.gz
    12:12:21.694 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_5/1.g.vcf.gz
    12:12:21.769 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_6/1.g.vcf.gz
    12:12:21.873 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_7/1.g.vcf.gz
    12:12:21.960 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_8/1.g.vcf.gz
    12:12:22.049 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_1/1.g.vcf.gz
    12:12:22.146 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_2/1.g.vcf.gz
    12:12:22.230 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_3/1.g.vcf.gz
    12:12:22.336 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_4/1.g.vcf.gz
    12:12:22.416 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_5/1.g.vcf.gz
    12:12:22.499 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_6/1.g.vcf.gz
    12:12:22.588 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_7/1.g.vcf.gz
    12:12:22.664 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_8/1.g.vcf.gz
    12:12:22.748 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16300_1/1.g.vcf.gz
    12:12:22.855 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16300_2/1.g.vcf.gz
    12:12:22.953 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16300_4/1.g.vcf.gz
    12:12:23.045 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_1/1.g.vcf.gz
    12:12:23.142 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_2/1.g.vcf.gz
    12:12:23.238 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_3/1.g.vcf.gz
    12:12:23.347 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_4/1.g.vcf.gz
    12:12:23.511 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_5/1.g.vcf.gz
    12:12:23.615 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_6/1.g.vcf.gz
    12:12:23.729 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_7/1.g.vcf.gz
    12:12:23.834 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_8/1.g.vcf.gz
    12:12:23.950 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_1/1.g.vcf.gz
    12:12:24.032 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_2/1.g.vcf.gz
    12:12:24.125 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_3/1.g.vcf.gz
    12:12:24.497 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_4/1.g.vcf.gz
    12:12:24.586 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_5/1.g.vcf.gz
    12:12:24.688 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_6/1.g.vcf.gz
    12:12:24.807 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_7/1.g.vcf.gz
    12:12:24.894 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_8/1.g.vcf.gz
    12:12:24.989 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_1/1.g.vcf.gz
    12:12:25.084 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_2/1.g.vcf.gz
    12:12:25.181 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_3/1.g.vcf.gz
    12:12:25.291 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_4/1.g.vcf.gz
    12:12:25.381 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_5/1.g.vcf.gz
    12:12:25.467 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_6/1.g.vcf.gz
    12:12:25.561 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_7/1.g.vcf.gz
    12:12:25.648 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_8/1.g.vcf.gz
    12:12:25.762 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_1/1.g.vcf.gz
    12:12:25.858 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_2/1.g.vcf.gz
    12:12:25.949 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_3/1.g.vcf.gz
    12:12:26.054 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_4/1.g.vcf.gz
    12:12:26.167 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_5/1.g.vcf.gz
    12:12:26.270 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_6/1.g.vcf.gz
    12:12:26.678 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_7/1.g.vcf.gz
    12:12:26.755 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_8/1.g.vcf.gz
    12:12:26.843 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_1/1.g.vcf.gz
    12:12:26.926 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_2/1.g.vcf.gz
    12:12:27.007 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_3/1.g.vcf.gz
    12:12:27.125 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_4/1.g.vcf.gz
    12:12:27.460 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_5/1.g.vcf.gz
    12:12:27.553 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_6/1.g.vcf.gz
    12:12:27.653 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_7/1.g.vcf.gz
    12:12:27.746 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_8/1.g.vcf.gz
    12:12:27.834 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_1/1.g.vcf.gz
    12:12:27.943 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_2/1.g.vcf.gz
    12:12:28.046 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_3/1.g.vcf.gz
    12:12:28.136 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_4/1.g.vcf.gz
    12:12:28.230 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_6/1.g.vcf.gz
    12:12:28.339 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_7/1.g.vcf.gz
    12:12:28.426 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_8/1.g.vcf.gz
    12:12:28.525 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_1/1.g.vcf.gz
    12:12:28.630 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_2/1.g.vcf.gz
    12:12:28.707 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_3/1.g.vcf.gz
    12:12:28.783 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_4/1.g.vcf.gz
    12:12:28.878 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_5/1.g.vcf.gz
    12:12:28.977 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_6/1.g.vcf.gz
    12:12:29.101 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_7/1.g.vcf.gz
    12:12:29.170 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_8/1.g.vcf.gz
    12:12:29.252 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_1/1.g.vcf.gz
    12:12:29.323 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_2/1.g.vcf.gz
    12:12:29.442 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_3/1.g.vcf.gz
    12:12:29.522 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_4/1.g.vcf.gz
    12:12:29.605 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_5/1.g.vcf.gz
    12:12:29.686 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_6/1.g.vcf.gz
    12:12:29.791 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_7/1.g.vcf.gz
    12:12:29.876 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_8/1.g.vcf.gz
    12:12:29.990 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16387_1/1.g.vcf.gz
    12:12:30.080 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16387_2/1.g.vcf.gz
    12:12:30.170 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/17193_1/1.g.vcf.gz
    12:12:30.252 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1%231/1.g.vcf.gz
    12:12:30.253 INFO  CombineGVCFs - Shutting down engine
    [02 February 2018 12:12:30 GMT] org.broadinstitute.hellbender.tools.walkers.CombineGVCFs done. Elapsed time: 0.30 minutes.
    Runtime.totalMemory()=2066743296
    org.broadinstitute.hellbender.exceptions.GATKException: Error initializing feature reader for path /lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1#1/1.g.vcf.gz
        at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:346)
        at org.broadinstitute.hellbender.engine.FeatureDataSource.getFeatureReader(FeatureDataSource.java:297)
        at org.broadinstitute.hellbender.engine.FeatureDataSource.<init>(FeatureDataSource.java:244)
        at org.broadinstitute.hellbender.engine.FeatureManager.addToFeatureSources(FeatureManager.java:202)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker.lambda$initializeDrivingVariants$0(MultiVariantWalker.java:66)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
        at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker.initializeDrivingVariants(MultiVariantWalker.java:56)
        at org.broadinstitute.hellbender.engine.VariantWalkerBase.initializeFeatures(VariantWalkerBase.java:47)
        at org.broadinstitute.hellbender.engine.GATKTool.onStartup(GATKTool.java:558)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker.onStartup(MultiVariantWalker.java:48)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:152)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:195)
        at org.broadinstitute.hellbender.Main.main(Main.java:275)
    Caused by: htsjdk.tribble.TribbleException$MalformedFeatureFile: Unable to create BasicFeatureReader using feature file , for input source: file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1%231/1.g.vcf.gz
        at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:113)
        at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:342)
        ... 16 more
    Caused by: java.io.FileNotFoundException: /lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1%231/1.g.vcf.gz (No such file or directory)
        at java.io.RandomAccessFile.open0(Native Method)
        at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
        at htsjdk.samtools.seekablestream.SeekableFileStream.<init>(SeekableFileStream.java:47)
        at htsjdk.samtools.seekablestream.SeekableStreamFactory$DefaultSeekableStreamFactory.getStreamFor(SeekableStreamFactory.java:99)
        at htsjdk.tribble.readers.TabixReader.<init>(TabixReader.java:129)
        at htsjdk.tribble.TabixFeatureReader.<init>(TabixFeatureReader.java:80)
        at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:106)
        ... 17 more
    
  • shleeshlee CambridgeMember, Broadie, Moderator

    @TechnicalVault, do you have indexes for these .vcf.gz files? There is a bug that makes .vcf.gz files (but not .vcf files) appear truncated unless accompanied by an index placed in the same directory. You can generate index files with IndexFeatureFile.

  • TechnicalVaultTechnicalVault Cambridge, UKMember

    I do indeed, they were generated by haplotype caller. The problem is the FileNotFoundException which seems to be caused by GATK internally UrlEncoding the # in the filename into a %23 when converting it to a file:/// URL and then forgetting to convert it back before feeding it to java's file API.

    Issue · Github
    by Sheila

    Issue Number
    4343
    State
    open
    Last Updated
    Assignee
    Array
    Milestone
    Array
  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @TechnicalVault
    Hi,

    I will ask the team and get back to you.

    -Sheila

Sign In or Register to comment.