To celebrate the release of GATK 4.0, we are giving away free credits for running the GATK4 Best Practices pipelines in FireCloud, our secure online analysis portal. It’s first come first serve, so sign up now to claim your free credits worth $250. Sponsored by Google Cloud. Learn more at https://software.broadinstitute.org/firecloud/documentation/freecredits

# in file names converted to %23 resulting in file not found

TechnicalVaultTechnicalVault Cambridge, UKMember
edited January 31 in Ask the GATK team

Whilst I was trying to run CombineGVCFs 4.0.0 I got a very strange error, file not found for a file I knew existed. Looking into the backtrace it looks like somehow a # is getting mistakenly URL escaped?

org.broadinstitute.hellbender.exceptions.GATKException: Error initializing feature reader for path /project/gvcf-pcr/23232_1#1/1.g.vcf.gz
    at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:341)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.getFeatureReader(FeatureDataSource.java:292)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.<init>(FeatureDataSource.java:244)
    at org.broadinstitute.hellbender.engine.FeatureManager.addToFeatureSources(FeatureManager.java:202)
    at org.broadinstitute.hellbender.engine.MultiVariantWalker.lambda$initializeDrivingVariants$0(MultiVariantWalker.java:66)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
    at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
    at org.broadinstitute.hellbender.engine.MultiVariantWalker.initializeDrivingVariants(MultiVariantWalker.java:56)
    at org.broadinstitute.hellbender.engine.VariantWalkerBase.initializeFeatures(VariantWalkerBase.java:47)
    at org.broadinstitute.hellbender.engine.GATKTool.onStartup(GATKTool.java:558)
    at org.broadinstitute.hellbender.engine.MultiVariantWalker.onStartup(MultiVariantWalker.java:48)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:152)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:195)
    at org.broadinstitute.hellbender.Main.main(Main.java:275)
Caused by: htsjdk.tribble.TribbleException$MalformedFeatureFile: Unable to create BasicFeatureReader using feature file , for input source: file:///project/gvcf-pcr/23232_1%231/1.g.vcf.gz
    at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:113)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:337)
    ... 16 more
Caused by: java.io.FileNotFoundException: /project/gvcf-pcr/23232_1%231/1.g.vcf.gz (No such file or directory)
    at java.io.RandomAccessFile.open0(Native Method)
    at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
    at htsjdk.samtools.seekablestream.SeekableFileStream.<init>(SeekableFileStream.java:47)
    at htsjdk.samtools.seekablestream.SeekableStreamFactory$DefaultSeekableStreamFactory.getStreamFor(SeekableStreamFactory.java:99)
    at htsjdk.tribble.readers.TabixReader.<init>(TabixReader.java:129)
    at htsjdk.tribble.TabixFeatureReader.<init>(TabixFeatureReader.java:83)
    at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:106)
    ... 17 more

Issue · Github
by Sheila

Issue Number
2894
State
closed
Last Updated
Assignee
Array
Closed By
chandrans

Best Answer

Answers

  • TechnicalVaultTechnicalVault Cambridge, UKMember

    Replicated in GATK 4.0.1.0 :(

  • shleeshlee CambridgeMember, Broadie, Moderator

    Please post your exact command that produced the error @TechnicalVault.

  • TechnicalVaultTechnicalVault Cambridge, UKMember
    edited February 2

    The command line and full run log was as follows, the tmp file was just a list of paths to the .g.vcf.gz files.

    The file in question was listed as
    /lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1#1/1.g.vcf.gz

    Using GATK jar /lustre/scratch115/realdata/mdt2/projects/gdap-wgs/gvcf-4.0/scripts/gatk-4.0.1.0/gatk-package-4.0.1.0-local.jar
    Running:
        /software/jre1.8.0_74/bin/java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -Djava.io.tmpdir=/lustre/scratch115/projects/gdap-wgs/gvcf-4.0/tmp -XX:-UsePerfData -Xrs -Xmx3200m -jar /lustre/scratch115/realdata/mdt2/projects/gdap-wgs/gvcf-4.0/scripts/gatk-4.0.1.0/gatk-package-4.0.1.0-local.jar CombineGVCFs -R /lustre/scratch115/resources/ref/Homo_sapiens/HS38DH/hs38DH.fa -V /tmp/tmp.7dp5SuO2TD.list -O /lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr_combined/1_1.g.vcf.gz -L /lustre/scratch115/resources/ref/Homo_sapiens/HS38DH/intervals/arvados/wgs_calling_regions.hg38.interval_list.1_of_200.interval_list
    12:12:12.775 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/lustre/scratch115/realdata/mdt2/projects/gdap-wgs/gvcf-4.0/scripts/gatk-4.0.1.0/gatk-package-4.0.1.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
    12:12:13.728 INFO  CombineGVCFs - ------------------------------------------------------------
    12:12:13.728 INFO  CombineGVCFs - The Genome Analysis Toolkit (GATK) v4.0.1.0
    12:12:13.729 INFO  CombineGVCFs - For support and documentation go to https://software.broadinstitute.org/gatk/
    12:12:13.729 INFO  CombineGVCFs - Executing as mp15@bc-25-1-04 on Linux v3.2.0-105-generic amd64
    12:12:13.730 INFO  CombineGVCFs - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_74-b02
    12:12:13.730 INFO  CombineGVCFs - Start Date/Time: 02 February 2018 12:12:12 GMT
    12:12:13.731 INFO  CombineGVCFs - ------------------------------------------------------------
    12:12:13.731 INFO  CombineGVCFs - ------------------------------------------------------------
    12:12:13.732 INFO  CombineGVCFs - HTSJDK Version: 2.14.1
    12:12:13.732 INFO  CombineGVCFs - Picard Version: 2.17.2
    12:12:13.732 INFO  CombineGVCFs - HTSJDK Defaults.COMPRESSION_LEVEL : 1
    12:12:13.733 INFO  CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    12:12:13.733 INFO  CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    12:12:13.733 INFO  CombineGVCFs - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    12:12:13.733 INFO  CombineGVCFs - Deflater: IntelDeflater
    12:12:13.733 INFO  CombineGVCFs - Inflater: IntelInflater
    12:12:13.734 INFO  CombineGVCFs - GCS max retries/reopens: 20
    12:12:13.734 INFO  CombineGVCFs - Using google-cloud-java patch 6d11bef1c81f885c26b2b56c8616b7a705171e4f from https://github.com/droazen/google-cloud-java/tree/dr_all_nio_fixes
    12:12:13.734 INFO  CombineGVCFs - Initializing engine
    12:12:16.228 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13939_6/1.g.vcf.gz
    12:12:16.453 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13939_7/1.g.vcf.gz
    12:12:16.582 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13939_8/1.g.vcf.gz
    12:12:16.735 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13940_5/1.g.vcf.gz
    12:12:16.833 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13940_6/1.g.vcf.gz
    12:12:16.975 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13940_7/1.g.vcf.gz
    12:12:17.124 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13940_8/1.g.vcf.gz
    12:12:17.229 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/13964_1/1.g.vcf.gz
    12:12:17.336 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/14538_6/1.g.vcf.gz
    12:12:17.427 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16006_4/1.g.vcf.gz
    12:12:17.558 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16006_5/1.g.vcf.gz
    12:12:17.651 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16006_6/1.g.vcf.gz
    12:12:17.756 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_1/1.g.vcf.gz
    12:12:17.878 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_2/1.g.vcf.gz
    12:12:17.985 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_3/1.g.vcf.gz
    12:12:18.086 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_4/1.g.vcf.gz
    12:12:18.180 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_5/1.g.vcf.gz
    12:12:18.319 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_6/1.g.vcf.gz
    12:12:18.444 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16007_7/1.g.vcf.gz
    12:12:18.565 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16014_4/1.g.vcf.gz
    12:12:18.675 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16014_6/1.g.vcf.gz
    12:12:18.771 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16014_7/1.g.vcf.gz
    12:12:18.892 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16015_1/1.g.vcf.gz
    12:12:18.996 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16015_3/1.g.vcf.gz
    12:12:19.116 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16015_4/1.g.vcf.gz
    12:12:19.232 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16015_7/1.g.vcf.gz
    12:12:19.399 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_4/1.g.vcf.gz
    12:12:19.487 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_5/1.g.vcf.gz
    12:12:19.572 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_6/1.g.vcf.gz
    12:12:19.762 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_7/1.g.vcf.gz
    12:12:19.856 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16062_8/1.g.vcf.gz
    12:12:19.961 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16063_3/1.g.vcf.gz
    12:12:20.051 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16063_4/1.g.vcf.gz
    12:12:20.130 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16063_6/1.g.vcf.gz
    12:12:20.248 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16063_7/1.g.vcf.gz
    12:12:20.327 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16279_1/1.g.vcf.gz
    12:12:20.420 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16279_2/1.g.vcf.gz
    12:12:20.554 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16279_3/1.g.vcf.gz
    12:12:20.652 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_1/1.g.vcf.gz
    12:12:20.739 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_2/1.g.vcf.gz
    12:12:20.823 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_3/1.g.vcf.gz
    12:12:20.920 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_4/1.g.vcf.gz
    12:12:21.013 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_5/1.g.vcf.gz
    12:12:21.114 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_6/1.g.vcf.gz
    12:12:21.199 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_7/1.g.vcf.gz
    12:12:21.270 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16280_8/1.g.vcf.gz
    12:12:21.351 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_1/1.g.vcf.gz
    12:12:21.438 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_2/1.g.vcf.gz
    12:12:21.544 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_3/1.g.vcf.gz
    12:12:21.611 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_4/1.g.vcf.gz
    12:12:21.694 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_5/1.g.vcf.gz
    12:12:21.769 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_6/1.g.vcf.gz
    12:12:21.873 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_7/1.g.vcf.gz
    12:12:21.960 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16281_8/1.g.vcf.gz
    12:12:22.049 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_1/1.g.vcf.gz
    12:12:22.146 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_2/1.g.vcf.gz
    12:12:22.230 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_3/1.g.vcf.gz
    12:12:22.336 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_4/1.g.vcf.gz
    12:12:22.416 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_5/1.g.vcf.gz
    12:12:22.499 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_6/1.g.vcf.gz
    12:12:22.588 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_7/1.g.vcf.gz
    12:12:22.664 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16299_8/1.g.vcf.gz
    12:12:22.748 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16300_1/1.g.vcf.gz
    12:12:22.855 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16300_2/1.g.vcf.gz
    12:12:22.953 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16300_4/1.g.vcf.gz
    12:12:23.045 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_1/1.g.vcf.gz
    12:12:23.142 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_2/1.g.vcf.gz
    12:12:23.238 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_3/1.g.vcf.gz
    12:12:23.347 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_4/1.g.vcf.gz
    12:12:23.511 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_5/1.g.vcf.gz
    12:12:23.615 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_6/1.g.vcf.gz
    12:12:23.729 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_7/1.g.vcf.gz
    12:12:23.834 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16303_8/1.g.vcf.gz
    12:12:23.950 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_1/1.g.vcf.gz
    12:12:24.032 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_2/1.g.vcf.gz
    12:12:24.125 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_3/1.g.vcf.gz
    12:12:24.497 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_4/1.g.vcf.gz
    12:12:24.586 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_5/1.g.vcf.gz
    12:12:24.688 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_6/1.g.vcf.gz
    12:12:24.807 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_7/1.g.vcf.gz
    12:12:24.894 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16305_8/1.g.vcf.gz
    12:12:24.989 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_1/1.g.vcf.gz
    12:12:25.084 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_2/1.g.vcf.gz
    12:12:25.181 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_3/1.g.vcf.gz
    12:12:25.291 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_4/1.g.vcf.gz
    12:12:25.381 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_5/1.g.vcf.gz
    12:12:25.467 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_6/1.g.vcf.gz
    12:12:25.561 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_7/1.g.vcf.gz
    12:12:25.648 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16306_8/1.g.vcf.gz
    12:12:25.762 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_1/1.g.vcf.gz
    12:12:25.858 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_2/1.g.vcf.gz
    12:12:25.949 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_3/1.g.vcf.gz
    12:12:26.054 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_4/1.g.vcf.gz
    12:12:26.167 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_5/1.g.vcf.gz
    12:12:26.270 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_6/1.g.vcf.gz
    12:12:26.678 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_7/1.g.vcf.gz
    12:12:26.755 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16307_8/1.g.vcf.gz
    12:12:26.843 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_1/1.g.vcf.gz
    12:12:26.926 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_2/1.g.vcf.gz
    12:12:27.007 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_3/1.g.vcf.gz
    12:12:27.125 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_4/1.g.vcf.gz
    12:12:27.460 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_5/1.g.vcf.gz
    12:12:27.553 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_6/1.g.vcf.gz
    12:12:27.653 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_7/1.g.vcf.gz
    12:12:27.746 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16328_8/1.g.vcf.gz
    12:12:27.834 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_1/1.g.vcf.gz
    12:12:27.943 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_2/1.g.vcf.gz
    12:12:28.046 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_3/1.g.vcf.gz
    12:12:28.136 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_4/1.g.vcf.gz
    12:12:28.230 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_6/1.g.vcf.gz
    12:12:28.339 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_7/1.g.vcf.gz
    12:12:28.426 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16329_8/1.g.vcf.gz
    12:12:28.525 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_1/1.g.vcf.gz
    12:12:28.630 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_2/1.g.vcf.gz
    12:12:28.707 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_3/1.g.vcf.gz
    12:12:28.783 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_4/1.g.vcf.gz
    12:12:28.878 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_5/1.g.vcf.gz
    12:12:28.977 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_6/1.g.vcf.gz
    12:12:29.101 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_7/1.g.vcf.gz
    12:12:29.170 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16331_8/1.g.vcf.gz
    12:12:29.252 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_1/1.g.vcf.gz
    12:12:29.323 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_2/1.g.vcf.gz
    12:12:29.442 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_3/1.g.vcf.gz
    12:12:29.522 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_4/1.g.vcf.gz
    12:12:29.605 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_5/1.g.vcf.gz
    12:12:29.686 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_6/1.g.vcf.gz
    12:12:29.791 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_7/1.g.vcf.gz
    12:12:29.876 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16332_8/1.g.vcf.gz
    12:12:29.990 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16387_1/1.g.vcf.gz
    12:12:30.080 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/16387_2/1.g.vcf.gz
    12:12:30.170 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/17193_1/1.g.vcf.gz
    12:12:30.252 INFO  FeatureManager - Using codec VCFCodec to read file file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1%231/1.g.vcf.gz
    12:12:30.253 INFO  CombineGVCFs - Shutting down engine
    [02 February 2018 12:12:30 GMT] org.broadinstitute.hellbender.tools.walkers.CombineGVCFs done. Elapsed time: 0.30 minutes.
    Runtime.totalMemory()=2066743296
    org.broadinstitute.hellbender.exceptions.GATKException: Error initializing feature reader for path /lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1#1/1.g.vcf.gz
        at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:346)
        at org.broadinstitute.hellbender.engine.FeatureDataSource.getFeatureReader(FeatureDataSource.java:297)
        at org.broadinstitute.hellbender.engine.FeatureDataSource.<init>(FeatureDataSource.java:244)
        at org.broadinstitute.hellbender.engine.FeatureManager.addToFeatureSources(FeatureManager.java:202)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker.lambda$initializeDrivingVariants$0(MultiVariantWalker.java:66)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
        at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker.initializeDrivingVariants(MultiVariantWalker.java:56)
        at org.broadinstitute.hellbender.engine.VariantWalkerBase.initializeFeatures(VariantWalkerBase.java:47)
        at org.broadinstitute.hellbender.engine.GATKTool.onStartup(GATKTool.java:558)
        at org.broadinstitute.hellbender.engine.MultiVariantWalker.onStartup(MultiVariantWalker.java:48)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:134)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:179)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:198)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:152)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:195)
        at org.broadinstitute.hellbender.Main.main(Main.java:275)
    Caused by: htsjdk.tribble.TribbleException$MalformedFeatureFile: Unable to create BasicFeatureReader using feature file , for input source: file:///lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1%231/1.g.vcf.gz
        at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:113)
        at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:342)
        ... 16 more
    Caused by: java.io.FileNotFoundException: /lustre/scratch115/projects/gdap-wgs/gvcf-4.0/gvcf-pcr/23232_1%231/1.g.vcf.gz (No such file or directory)
        at java.io.RandomAccessFile.open0(Native Method)
        at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
        at htsjdk.samtools.seekablestream.SeekableFileStream.<init>(SeekableFileStream.java:47)
        at htsjdk.samtools.seekablestream.SeekableStreamFactory$DefaultSeekableStreamFactory.getStreamFor(SeekableStreamFactory.java:99)
        at htsjdk.tribble.readers.TabixReader.<init>(TabixReader.java:129)
        at htsjdk.tribble.TabixFeatureReader.<init>(TabixFeatureReader.java:80)
        at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:106)
        ... 17 more
    
  • shleeshlee CambridgeMember, Broadie, Moderator

    @TechnicalVault, do you have indexes for these .vcf.gz files? There is a bug that makes .vcf.gz files (but not .vcf files) appear truncated unless accompanied by an index placed in the same directory. You can generate index files with IndexFeatureFile.

  • TechnicalVaultTechnicalVault Cambridge, UKMember

    I do indeed, they were generated by haplotype caller. The problem is the FileNotFoundException which seems to be caused by GATK internally UrlEncoding the # in the filename into a %23 when converting it to a file:/// URL and then forgetting to convert it back before feeding it to java's file API.

    Issue · Github
    by Sheila

    Issue Number
    4343
    State
    open
    Last Updated
    Assignee
    Array
    Milestone
    Array
  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @TechnicalVault
    Hi,

    I will ask the team and get back to you.

    -Sheila

Sign In or Register to comment.