We've moved!
This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!

GATK GenomicsDBImport error

Hi, I am trying to run gatk GenomicsDBI import and it is giving an error (see below). I seems like the tools runs fine and I can even use it DB with GenotypeGVCFs. I have validated all VCF files using ValidateVariants.

Any suggestions?

Thanks

gatk GenomicsDBImport --genomicsdb-workspace-path my_database --batch-size 500 -L Chromosome:3343176-3343820 --sample-name-map samples.map --reader-threads 40 --tmp-dir=. --java-options '-Xmx50g'

Using GATK jar /mnt/storage/jody/miniconda3/envs/dev/share/gatk4-4.1.3.0-0/gatk-package-4.1.3.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx50g -jar /mnt/storage/jody/miniconda3/envs/dev/share/gatk4-4.1.3.0-0/gatk-package-4.1.3.0-local.jar GenomicsDBImport --genomicsdb-workspace-path my_database --batch-size 500 -L Chromosome:3343176-3343820 --sample-name-map samples.map --reader-threads 40 --tmp-dir=.
01:44:53.437 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/mnt/storage/jody/miniconda3/envs/dev/share/gatk4-4.1.3.0-0/gatk-package-4.1.3.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
Sep 26, 2019 1:44:55 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
01:44:55.301 INFO GenomicsDBImport - ------------------------------------------------------------
01:44:55.302 INFO GenomicsDBImport - The Genome Analysis Toolkit (GATK) v4.1.3.0
01:44:55.302 INFO GenomicsDBImport - For support and documentation go to https://software.broadinstitute.org/gatk/
01:44:55.302 INFO GenomicsDBImport - Executing as [email protected] on Linux v4.2.0-27-generic amd64
01:44:55.302 INFO GenomicsDBImport - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_192-b01
01:44:55.303 INFO GenomicsDBImport - Start Date/Time: 26 September 2019 01:44:53 BST
01:44:55.303 INFO GenomicsDBImport - ------------------------------------------------------------
01:44:55.303 INFO GenomicsDBImport - ------------------------------------------------------------
01:44:55.304 INFO GenomicsDBImport - HTSJDK Version: 2.20.1
01:44:55.304 INFO GenomicsDBImport - Picard Version: 2.20.5
01:44:55.304 INFO GenomicsDBImport - HTSJDK Defaults.COMPRESSION_LEVEL : 2
01:44:55.304 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
01:44:55.304 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
01:44:55.304 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
01:44:55.304 INFO GenomicsDBImport - Deflater: IntelDeflater
01:44:55.304 INFO GenomicsDBImport - Inflater: IntelInflater
01:44:55.305 INFO GenomicsDBImport - GCS max retries/reopens: 20
01:44:55.305 INFO GenomicsDBImport - Requester pays: disabled
01:44:55.305 INFO GenomicsDBImport - Initializing engine
01:44:55.779 INFO IntervalArgumentCollection - Processing 645 bp from intervals
01:44:55.780 INFO GenomicsDBImport - Done initializing engine
01:44:55.976 INFO GenomicsDBImport - Vid Map JSON file will be written to /mnt/storage/jody/ena/leopold/my_database/vidmap.json
01:44:55.976 INFO GenomicsDBImport - Callset Map JSON file will be written to /mnt/storage/jody/ena/leopold/my_database/callset.json
01:44:55.976 INFO GenomicsDBImport - Complete VCF Header will be written to /mnt/storage/jody/ena/leopold/my_database/vcfheader.vcf
01:44:55.976 INFO GenomicsDBImport - Importing to array - /mnt/storage/jody/ena/leopold/my_database/genomicsdb_array
01:44:55.977 INFO ProgressMeter - Starting traversal
01:44:55.977 INFO ProgressMeter - Current Locus Elapsed Minutes Batches Processed Batches/Minute
01:44:56.289 INFO GenomicsDBImport - Starting batch input file preload
01:44:58.037 INFO GenomicsDBImport - Finished batch preload
01:44:58.037 INFO GenomicsDBImport - Importing batch 1 with 500 samples
01:44:59.890 INFO GenomicsDBImport - Done importing batch 1/36
01:44:59.890 INFO GenomicsDBImport - Starting batch input file preload
01:45:00.863 INFO GenomicsDBImport - Finished batch preload
01:45:00.863 INFO GenomicsDBImport - Importing batch 2 with 500 samples
01:45:04.317 INFO GenomicsDBImport - Done importing batch 2/36
01:45:04.317 INFO GenomicsDBImport - Starting batch input file preload
01:45:05.236 INFO GenomicsDBImport - Finished batch preload
01:45:05.236 INFO GenomicsDBImport - Importing batch 3 with 500 samples
01:45:06.552 INFO ProgressMeter - Chromosome:3343176 0.2 3 17.0
01:45:06.552 INFO GenomicsDBImport - Done importing batch 3/36
01:45:06.553 INFO GenomicsDBImport - Starting batch input file preload
01:45:06.846 INFO GenomicsDBImport - Finished batch preload
01:45:06.846 INFO GenomicsDBImport - Importing batch 4 with 500 samples
01:45:08.196 INFO GenomicsDBImport - Done importing batch 4/36
01:45:08.196 INFO GenomicsDBImport - Starting batch input file preload
01:45:09.411 INFO GenomicsDBImport - Finished batch preload
01:45:09.411 INFO GenomicsDBImport - Importing batch 5 with 500 samples
01:45:10.575 INFO GenomicsDBImport - Done importing batch 5/36
01:45:10.575 INFO GenomicsDBImport - Starting batch input file preload
01:45:10.934 INFO GenomicsDBImport - Finished batch preload
01:45:10.934 INFO GenomicsDBImport - Importing batch 6 with 500 samples
01:45:12.244 INFO GenomicsDBImport - Done importing batch 6/36
01:45:12.244 INFO GenomicsDBImport - Starting batch input file preload
01:45:12.915 INFO GenomicsDBImport - Finished batch preload
01:45:12.915 INFO GenomicsDBImport - Importing batch 7 with 500 samples
01:45:13.777 INFO GenomicsDBImport - Done importing batch 7/36
01:45:13.777 INFO GenomicsDBImport - Starting batch input file preload
01:45:14.132 INFO GenomicsDBImport - Finished batch preload
01:45:14.132 INFO GenomicsDBImport - Importing batch 8 with 500 samples
01:45:17.730 INFO ProgressMeter - Chromosome:3343176 0.4 8 22.1
01:45:17.731 INFO GenomicsDBImport - Done importing batch 8/36
01:45:17.732 INFO GenomicsDBImport - Starting batch input file preload
01:45:19.248 INFO GenomicsDBImport - Finished batch preload
01:45:19.248 INFO GenomicsDBImport - Importing batch 9 with 500 samples
01:45:20.292 INFO GenomicsDBImport - Done importing batch 9/36
01:45:20.293 INFO GenomicsDBImport - Starting batch input file preload
01:45:20.512 INFO GenomicsDBImport - Finished batch preload
01:45:20.513 INFO GenomicsDBImport - Importing batch 10 with 500 samples
01:45:22.159 INFO GenomicsDBImport - Done importing batch 10/36
01:45:22.159 INFO GenomicsDBImport - Starting batch input file preload
01:45:23.519 INFO GenomicsDBImport - Finished batch preload
01:45:23.519 INFO GenomicsDBImport - Importing batch 11 with 500 samples
01:45:25.032 INFO GenomicsDBImport - Done importing batch 11/36
01:45:25.033 INFO GenomicsDBImport - Starting batch input file preload
01:45:25.360 INFO GenomicsDBImport - Finished batch preload
01:45:25.360 INFO GenomicsDBImport - Importing batch 12 with 500 samples
01:45:26.533 INFO GenomicsDBImport - Done importing batch 12/36
01:45:26.534 INFO GenomicsDBImport - Starting batch input file preload
01:45:27.308 INFO GenomicsDBImport - Finished batch preload
01:45:27.308 INFO GenomicsDBImport - Importing batch 13 with 500 samples
01:45:31.119 INFO ProgressMeter - Chromosome:3343176 0.6 13 22.2
01:45:31.119 INFO GenomicsDBImport - Done importing batch 13/36
01:45:31.119 INFO GenomicsDBImport - Starting batch input file preload
01:45:32.688 INFO GenomicsDBImport - Finished batch preload
01:45:32.689 INFO GenomicsDBImport - Importing batch 14 with 500 samples
01:45:34.246 INFO GenomicsDBImport - Done importing batch 14/36
01:45:34.246 INFO GenomicsDBImport - Starting batch input file preload
01:45:34.589 INFO GenomicsDBImport - Finished batch preload
01:45:34.589 INFO GenomicsDBImport - Importing batch 15 with 500 samples
01:45:36.108 INFO GenomicsDBImport - Done importing batch 15/36
01:45:36.109 INFO GenomicsDBImport - Starting batch input file preload
01:45:36.613 INFO GenomicsDBImport - Finished batch preload
01:45:36.613 INFO GenomicsDBImport - Importing batch 16 with 500 samples
01:45:38.998 INFO GenomicsDBImport - Done importing batch 16/36
01:45:38.998 INFO GenomicsDBImport - Starting batch input file preload
01:45:39.362 INFO GenomicsDBImport - Finished batch preload
01:45:39.362 INFO GenomicsDBImport - Importing batch 17 with 500 samples
01:45:40.850 INFO GenomicsDBImport - Done importing batch 17/36
01:45:40.850 INFO GenomicsDBImport - Starting batch input file preload
01:45:41.249 INFO GenomicsDBImport - Finished batch preload
01:45:41.249 INFO GenomicsDBImport - Importing batch 18 with 500 samples
01:45:42.568 INFO ProgressMeter - Chromosome:3343176 0.8 18 23.2
01:45:42.568 INFO GenomicsDBImport - Done importing batch 18/36
01:45:42.568 INFO GenomicsDBImport - Starting batch input file preload
01:45:43.812 INFO GenomicsDBImport - Finished batch preload
01:45:43.812 INFO GenomicsDBImport - Importing batch 19 with 500 samples
01:45:45.008 INFO GenomicsDBImport - Done importing batch 19/36
01:45:45.008 INFO GenomicsDBImport - Starting batch input file preload
01:45:45.224 INFO GenomicsDBImport - Finished batch preload
01:45:45.224 INFO GenomicsDBImport - Importing batch 20 with 500 samples
01:45:46.446 INFO GenomicsDBImport - Done importing batch 20/36
01:45:46.447 INFO GenomicsDBImport - Starting batch input file preload
01:45:46.754 INFO GenomicsDBImport - Finished batch preload
01:45:46.754 INFO GenomicsDBImport - Importing batch 21 with 500 samples
01:45:47.967 INFO GenomicsDBImport - Done importing batch 21/36
01:45:47.967 INFO GenomicsDBImport - Starting batch input file preload
01:45:48.685 INFO GenomicsDBImport - Finished batch preload
01:45:48.685 INFO GenomicsDBImport - Importing batch 22 with 500 samples
01:45:49.785 INFO GenomicsDBImport - Done importing batch 22/36
01:45:49.785 INFO GenomicsDBImport - Starting batch input file preload
01:45:49.906 INFO GenomicsDBImport - Finished batch preload
01:45:49.906 INFO GenomicsDBImport - Importing batch 23 with 500 samples
01:45:51.206 INFO GenomicsDBImport - Done importing batch 23/36
01:45:51.207 INFO GenomicsDBImport - Starting batch input file preload
01:45:51.383 INFO GenomicsDBImport - Finished batch preload
01:45:51.384 INFO GenomicsDBImport - Importing batch 24 with 500 samples
01:45:52.789 INFO ProgressMeter - Chromosome:3343176 0.9 24 25.3
01:45:52.789 INFO GenomicsDBImport - Done importing batch 24/36
01:45:52.789 INFO GenomicsDBImport - Starting batch input file preload
01:45:53.052 INFO GenomicsDBImport - Finished batch preload
01:45:53.052 INFO GenomicsDBImport - Importing batch 25 with 500 samples
01:45:54.187 INFO GenomicsDBImport - Done importing batch 25/36
01:45:54.187 INFO GenomicsDBImport - Starting batch input file preload
01:45:54.516 INFO GenomicsDBImport - Finished batch preload
01:45:54.516 INFO GenomicsDBImport - Importing batch 26 with 500 samples
01:45:55.719 INFO GenomicsDBImport - Done importing batch 26/36
01:45:55.720 INFO GenomicsDBImport - Starting batch input file preload
01:45:56.041 INFO GenomicsDBImport - Finished batch preload
01:45:56.041 INFO GenomicsDBImport - Importing batch 27 with 500 samples
01:45:57.172 INFO GenomicsDBImport - Done importing batch 27/36
01:45:57.172 INFO GenomicsDBImport - Starting batch input file preload
01:45:57.828 INFO GenomicsDBImport - Finished batch preload
01:45:57.828 INFO GenomicsDBImport - Importing batch 28 with 500 samples
01:45:58.832 INFO GenomicsDBImport - Done importing batch 28/36
01:45:58.832 INFO GenomicsDBImport - Starting batch input file preload
01:45:59.038 INFO GenomicsDBImport - Finished batch preload
01:45:59.038 INFO GenomicsDBImport - Importing batch 29 with 500 samples
01:46:00.158 INFO GenomicsDBImport - Done importing batch 29/36
01:46:00.159 INFO GenomicsDBImport - Starting batch input file preload
01:46:00.387 INFO GenomicsDBImport - Finished batch preload
01:46:00.388 INFO GenomicsDBImport - Importing batch 30 with 500 samples
01:46:01.252 INFO GenomicsDBImport - Done importing batch 30/36
01:46:01.253 INFO GenomicsDBImport - Starting batch input file preload
01:46:01.884 INFO GenomicsDBImport - Finished batch preload
01:46:01.884 INFO GenomicsDBImport - Importing batch 31 with 500 samples
01:46:03.322 INFO ProgressMeter - Chromosome:3343176 1.1 31 27.6
01:46:03.323 INFO GenomicsDBImport - Done importing batch 31/36
01:46:03.323 INFO GenomicsDBImport - Starting batch input file preload
01:46:03.688 INFO GenomicsDBImport - Finished batch preload
01:46:03.688 INFO GenomicsDBImport - Importing batch 32 with 500 samples
01:46:05.148 INFO GenomicsDBImport - Done importing batch 32/36
01:46:05.148 INFO GenomicsDBImport - Starting batch input file preload
01:46:05.706 INFO GenomicsDBImport - Finished batch preload
01:46:05.706 INFO GenomicsDBImport - Importing batch 33 with 500 samples
01:46:07.155 INFO GenomicsDBImport - Done importing batch 33/36
01:46:07.156 INFO GenomicsDBImport - Starting batch input file preload
01:46:07.456 INFO GenomicsDBImport - Finished batch preload
01:46:07.456 INFO GenomicsDBImport - Importing batch 34 with 500 samples
01:46:09.047 INFO GenomicsDBImport - Done importing batch 34/36
01:46:09.047 INFO GenomicsDBImport - Starting batch input file preload
01:46:15.085 INFO GenomicsDBImport - Shutting down engine
[26 September 2019 01:46:15 BST] org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport done. Elapsed time: 1.36 minutes.
Runtime.totalMemory()=28423225344


A USER ERROR has occurred: Couldn't read file. Error was: Failure while waiting for FeatureReader to initialize with exception: htsjdk.samtools.SAMFormatException: Invalid GZIP header


Answers

Sign In or Register to comment.