Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.

ImportGVCFs error message "Failed to create reader from file"

Hello,

I am using joint-discovery-gatk4.wdl + joint-discovery-gatk4.hg38.wgs.inputs.json to run “Generic germline short variant joint genotyping” pipeline with GATK 4.0.3.0 on my local computer.
I got an error message: “A USER ERROR has occurred: Failed to create reader from file:///home/carl/temp/NA12878.g.vcf.gz” when running the task “ImportGVCFs”. What dose this message mean and how about the solution?

**Here is the content of cromwell-executions/JointGenotyping/…/call-ImportGVCFs/shard-1/execution/sterr: **

Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell-executions/JointGenotyping/e584fd7e-1bb8-46ee-8daa-5630351324c2/call-ImportGVCFs/shard-1/tmp.ad3e936f
13:39:13.501 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/build/libs/gatk-package-4.0.3.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
13:39:13.723 INFO GenomicsDBImport - ------------------------------------------------------------
13:39:13.723 INFO GenomicsDBImport - The Genome Analysis Toolkit (GATK) v4.0.3.0
13:39:13.723 INFO GenomicsDBImport - For support and documentation go to https://software.broadinstitute.org/gatk/
13:39:13.724 INFO GenomicsDBImport - Executing as [email protected] on Linux v4.13.0-38-generic amd64
13:39:13.724 INFO GenomicsDBImport - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11
13:39:13.724 INFO GenomicsDBImport - Start Date/Time: April 11, 2018 1:39:13 PM UTC
13:39:13.724 INFO GenomicsDBImport - ------------------------------------------------------------
13:39:13.724 INFO GenomicsDBImport - ------------------------------------------------------------
13:39:13.725 INFO GenomicsDBImport - HTSJDK Version: 2.14.3
13:39:13.725 INFO GenomicsDBImport - Picard Version: 2.17.2
13:39:13.725 INFO GenomicsDBImport - HTSJDK Defaults.COMPRESSION_LEVEL : 2
13:39:13.725 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
13:39:13.725 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
13:39:13.725 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
13:39:13.725 INFO GenomicsDBImport - Deflater: IntelDeflater
13:39:13.725 INFO GenomicsDBImport - Inflater: IntelInflater
13:39:13.725 INFO GenomicsDBImport - GCS max retries/reopens: 20
13:39:13.725 INFO GenomicsDBImport - Using google-cloud-java patch 6d11bef1c81f885c26b2b56c8616b7a705171e4f from https://github.com/droazen/google-cloud-java/tree/dr_all_nio_fixes
13:39:13.725 INFO GenomicsDBImport - Initializing engine
13:39:13.747 INFO GenomicsDBImport - Shutting down engine
[April 11, 2018 1:39:13 PM UTC] org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=4116185088


A USER ERROR has occurred: Failed to create reader from file:///home/carl/temp/NA12878.g.vcf.gz


Set the system property GATK_STACKTRACE_ON_USER_EXCEPTION (--java-options '-DGATK_STACKTRACE_ON_USER_EXCEPTION=true') to print the stack trace.
Using GATK jar /gatk/build/libs/gatk-package-4.0.3.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx4g -Xms4g -jar /gatk/build/libs/gatk-package-4.0.3.0-local.jar GenomicsDBImport --genomicsdb-workspace-path genomicsdb --batch-size 50 -L chr2:1-242193529 --sample-name-map /cromwell-executions/JointGenotyping/e584fd7e-1bb8-46ee-8daa-5630351324c2/call-ImportGVCFs/shard-1/inputs/home/carl/WDL/NA12878.sample_map --reader-threads 5 -ip 500

**And here is the code of “joint-discovery-gatk4.hg38.wgs.inputs.json”: **

{
"##_COMMENT1": "INPUT GVCFs & COHORT -- DATASET-SPECIFC, MUST BE ADAPTED",
"JointGenotyping.callset_name": "NA12878",
"JointGenotyping.sample_name_map": "/home/carl/WDL/NA12878.sample_map",

"##_COMMENT2": "REFERENCE FILES",
"JointGenotyping.ref_fasta": "/home/carl/GATK4/hg38/Homo_sapiens_assembly38.fasta",
"JointGenotyping.ref_fasta_index": "/home/carl/GATK4/hg38/Homo_sapiens_assembly38.fasta.fai",
"JointGenotyping.ref_dict": "/home/carl/GATK4/hg38/Homo_sapiens_assembly38.dict",

"##_COMMENT3": "INTERVALS",
"JointGenotyping.eval_interval_list": "/home/carl/WDL/wgs_evaluation_regions.hg38.interval_list",
"JointGenotyping.unpadded_intervals_file": "/home/carl/WDL/hg38.even.handcurated.20k.intervals",

"##_COMMENT4": "RESOURCE FILES",
"JointGenotyping.dbsnp_vcf": "/home/carl/GATK4/hg38/Homo_sapiens_assembly38.dbsnp138.vcf",
"JointGenotyping.dbsnp_vcf_index": "/home/carl/GATK4/hg38/Homo_sapiens_assembly38.dbsnp138.vcf.idx",
"JointGenotyping.one_thousand_genomes_resource_vcf": "/home/carl/GATK4/hg38/1000G_phase1.snps.high_confidence.hg38.vcf.gz",
"JointGenotyping.one_thousand_genomes_resource_vcf_index": "/home/carl/GATK4/hg38/1000G_phase1.snps.high_confidence.hg38.vcf.gz.tbi",
"JointGenotyping.omni_resource_vcf": "/home/carl/GATK4/hg38/1000G_omni2.5.hg38.vcf.gz",
"JointGenotyping.omni_resource_vcf_index": "/home/carl/GATK4/hg38/1000G_omni2.5.hg38.vcf.gz.tbi",
"JointGenotyping.mills_resource_vcf": "/home/carl/GATK4/hg38/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz",
"JointGenotyping.mills_resource_vcf_index": "/home/carl/GATK4/hg38/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz.tbi",
"JointGenotyping.axiomPoly_resource_vcf": "/home/carl/GATK4/hg38/Axiom_Exome_Plus.genotypes.all_populations.poly.hg38.vcf.gz",
"JointGenotyping.axiomPoly_resource_vcf_index": "/home/carl/GATK4/hg38/Axiom_Exome_Plus.genotypes.all_populations.poly.hg38.vcf.gz.tbi",
"JointGenotyping.hapmap_resource_vcf": "/home/carl/GATK4/hg38/hapmap_3.3.hg38.vcf.gz",
"JointGenotyping.hapmap_resource_vcf_index": "/home/carl/GATK4/hg38/hapmap_3.3.hg38.vcf.gz.tbi",

"##_COMMENT5": "VQSR PARAMETERS",
"JointGenotyping.SNP_VQSR_downsampleFactor": 10,
"JointGenotyping.snp_filter_level": 99.7,
"JointGenotyping.indel_filter_level": 99.7,
"JointGenotyping.indel_recalibration_annotation_values": ["FS", "ReadPosRankSum", "MQRankSum", "QD", "SOR", "DP"],
"JointGenotyping.indel_recalibration_tranche_values": ["100.0", "99.95", "99.9", "99.5", "99.0", "97.0", "96.0", "95.0", "94.0", "93.5", "93.0", "92.0", "91.0", "90.0"],
"JointGenotyping.snp_recalibration_tranche_values": ["100.0", "99.95", "99.9", "99.8", "99.6", "99.5", "99.4", "99.3", "99.0", "98.0", "97.0", "90.0" ],
"JointGenotyping.snp_recalibration_annotation_values": ["QD", "MQRankSum", "ReadPosRankSum", "FS", "MQ", "SOR", "DP"],

"##_COMMENT4": "DOCKERS",
"JointGenotyping.python_docker": "python:2.7",
"JointGenotyping.gatk_docker": "broadinstitute/gatk:4.0.3.0",

"##_COMMENT5": "PATHS",
"JointGenotyping.gatk_path": "/gatk/gatk",

"##_COMMENT6": "JAVA OPTIONS",
"JointGenotyping.SNPsVariantRecalibratorScattered.java_opt": "-Xmx3g -Xms3g",
"JointGenotyping.CollectMetricsOnFullVcf.java_opt": "-Xmx6g -Xms6g",
"JointGenotyping.IndelsVariantRecalibrator.java_opt": "-Xmx24g -Xms24g",
"JointGenotyping.HardFilterAndMakeSitesOnlyVcf.java_opt": "-Xmx3g -Xms3g",
"JointGenotyping.SNPGatherTranches.java_opt": "-Xmx6g -Xms6g",
"JointGenotyping.CollectMetricsSharded.java_opt": "-Xmx6g -Xms6g",
"JointGenotyping.SitesOnlyGatherVcf.java_opt": "-Xmx6g -Xms6g",
"JointGenotyping.ApplyRecalibration.java_opt": "-Xmx5g -Xms5g",
"JointGenotyping.FinalGatherVcf.java_opt": "-Xmx6g -Xms6g",
"JointGenotyping.ImportGVCFs.java_opt": "-Xmx4g -Xms4g",
"JointGenotyping.SNPsVariantRecalibratorCreateModel.java_opt": "-Xmx100g -Xms100g",
"JointGenotyping.GatherMetrics.java_opt": "-Xmx2g -Xms2g",
"JointGenotyping.GenotypeGVCFs.java_opt": "-Xmx5g -Xms5g",

"##_COMMENT7": "MEMORY ALLOCATION",
"JointGenotyping.CollectMetricsSharded.mem_size": "7 GB",
"JointGenotyping.ImportGVCFs.mem_size": "7 GB",
"JointGenotyping.IndelsVariantRecalibrator.mem_size": "26 GB",
"JointGenotyping.ApplyRecalibration.mem_size": "7 GB",
"JointGenotyping.CollectMetricsOnFullVcf.mem_size": "7 GB",
"JointGenotyping.GenotypeGVCFs.mem_size": "7 GB",
"JointGenotyping.FinalGatherVcf.mem_size": "7 GB",
"JointGenotyping.SitesOnlyGatherVcf.mem_size": "7 GB",
"JointGenotyping.SNPsVariantRecalibratorScattered.mem_size": "3.5 GB",
"JointGenotyping.SNPsVariantRecalibratorCreateModel.mem_size": "104 GB",
"JointGenotyping.DynamicallyCombineIntervals.mem_size": "3 GB",
"JointGenotyping.GatherMetrics.mem_size": "3 GB",
"JointGenotyping.HardFilterAndMakeSitesOnlyVcf.mem_size": "3.5 GB",
"JointGenotyping.SNPGatherTranches.mem_size": "7 GB",

"##_COMMENT8": "DISK SIZE ALLOCATION",
"JointGenotyping.small_disk": 100,
"JointGenotyping.medium_disk": 200,
"JointGenotyping.huge_disk": 300,

"##_COMMENT9": "PREEMPTIBLES",
"JointGenotyping.FinalGatherVcf.preemptibles": 5,
"JointGenotyping.IndelsVariantRecalibrator.preemptibles": 5,
"JointGenotyping.SNPsVariantRecalibratorScattered.preemptibles": 5,
"JointGenotyping.HardFilterAndMakeSitesOnlyVcf.preemptibles": 5,
"JointGenotyping.CollectMetricsOnFullVcf.preemptibles": 5,
"JointGenotyping.GatherMetrics.preemptibles": 5,
"JointGenotyping.SNPGatherTranches.preemptibles": 5,
"JointGenotyping.SitesOnlyGatherVcf.preemptibles": 5,
"JointGenotyping.CollectMetricsSharded.preemptibles": 5,
"JointGenotyping.ApplyRecalibration.preemptibles": 5,
"JointGenotyping.ImportGVCFs.preemptibles": 5,
"JointGenotyping.SNPsVariantRecalibratorCreateModel.preemptibles": 5,
"JointGenotyping.GenotypeGVCFs.preemptibles": 5,
"JointGenotyping.DynamicallyCombineIntervals.preemptibles": 5
}

Here is the content of "/home/carl/WDL/NA12878.sample_map" : 

NA12878 /home/carl/temp/NA12878.g.vcf.gz

Best regards,
Carl

Best Answer

Answers

Sign In or Register to comment.