To celebrate the release of GATK 4.0, we are giving away free credits for running the GATK4 Best Practices pipelines in FireCloud, our secure online analysis portal. It’s first come first serve, so sign up now to claim your free credits worth $250. Sponsored by Google Cloud. Learn more at https://software.broadinstitute.org/firecloud/documentation/freecredits

Trouble with running GenomicsDBImport

I'm writing a pipeline using GATK4 for our local cluster which uses Slurm as job scheduler. The command below seems to run successfully, however, it took only a few seconds and the output file sizes are very small. Using the genomics db file from the output as input for joint genotyping, the output vcf only contains header section.

gatk --java-options "-Xmx8000M" GenomicsDBImport -V /gpfs/scratch/jw24/variant_discovery/raw_vcf/TEST/TEST_sample_2745_T_AS.g.vcf -V /gpfs/scratch/jw24/variant_discovery/raw_vcf/TEST/TEST_sample_2753_T_AS.g.vcf --genomicsdb-workspace-path /gpfs/scratch/jw24/variant_discovery/genomicsDB/chr1GenomicDB -L chr1
Using GATK jar /util/common/bioinformatics/GATK/gatk-4.0.0.0/gatk-package-4.0.0.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -Xmx8000M -jar /util/common/bioinformatics/GATK/gatk-4.0.0.0/gatk-package-4.0.0.0-local.jar GenomicsDBImport -V /gpfs/scratch/jw24/variant_discovery/raw_vcf/TEST/TEST_MMRF_2745_T_AS.g.vcf -V /gpfs/scratch/jw24/variant_discovery/raw_vcf/TEST/TEST_MMRF_2753_T_AS.g.vcf --genomicsdb-workspace-path /gpfs/scratch/jw24/variant_discovery/genomicsDB/chr1GenomicDB -L chr1
11:47:00.370 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/util/common/bioinformatics/GATK/gatk-4.0.0.0/gatk-package-4.0.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
11:47:00.540 INFO GenomicsDBImport - ------------------------------------------------------------
11:47:00.541 INFO GenomicsDBImport - The Genome Analysis Toolkit (GATK) v4.0.0.0
11:47:00.541 INFO GenomicsDBImport - For support and documentation go to https://software.broadinstitute.org/gatk/
11:47:00.541 INFO GenomicsDBImport - Executing as jw24@srv-p27-40.cbls.ccr.buffalo.edu on Linux v3.10.0-693.11.6.el7.x86_64 amd64
11:47:00.542 INFO GenomicsDBImport - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_45-b14
11:47:00.542 INFO GenomicsDBImport - Start Date/Time: February 5, 2018 11:47:00 AM EST
11:47:00.542 INFO GenomicsDBImport - ------------------------------------------------------------
11:47:00.542 INFO GenomicsDBImport - ------------------------------------------------------------
11:47:00.543 INFO GenomicsDBImport - HTSJDK Version: 2.13.2
11:47:00.543 INFO GenomicsDBImport - Picard Version: 2.17.2
11:47:00.543 INFO GenomicsDBImport - HTSJDK Defaults.COMPRESSION_LEVEL : 1
11:47:00.543 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
11:47:00.543 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
11:47:00.543 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
11:47:00.543 INFO GenomicsDBImport - Deflater: IntelDeflater
11:47:00.544 INFO GenomicsDBImport - Inflater: IntelInflater
11:47:00.544 INFO GenomicsDBImport - GCS max retries/reopens: 20
11:47:00.544 INFO GenomicsDBImport - Using google-cloud-java patch 6d11bef1c81f885c26b2b56c8616b7a705171e4f from https://github.com/droazen/google-cloud-java/tree/dr_all_nio_fixes
11:47:00.544 INFO GenomicsDBImport - Initializing engine
11:47:01.241 INFO IntervalArgumentCollection - Processing 248956422 bp from intervals
11:47:01.244 INFO GenomicsDBImport - Done initializing engine
Created workspace /gpfs/scratch/jw24/variant_discovery/genomicsDB/chr1GenomicDB
11:47:01.437 INFO GenomicsDBImport - Vid Map JSON file will be written to /gpfs/scratch/jw24/variant_discovery/genomicsDB/chr1GenomicDB/vidmap.json
11:47:01.437 INFO GenomicsDBImport - Callset Map JSON file will be written to /gpfs/scratch/jw24/variant_discovery/genomicsDB/chr1GenomicDB/callset.json
11:47:01.438 INFO GenomicsDBImport - Complete VCF Header will be written to /gpfs/scratch/jw24/variant_discovery/genomicsDB/chr1GenomicDB/vcfheader.vcf
11:47:01.438 INFO GenomicsDBImport - Importing to array - /gpfs/scratch/jw24/variant_discovery/genomicsDB/chr1GenomicDB/genomicsdb_array
11:47:01.456 INFO ProgressMeter - Starting traversal
11:47:01.457 INFO ProgressMeter - Current Locus Elapsed Minutes Batches Processed Batches/Minute
11:47:01.704 INFO GenomicsDBImport - Importing batch 1 with 2 samples
11:47:01.850 INFO GenomicsDBImport - Done importing batch 1/1
11:47:01.851 INFO ProgressMeter - chr1:1 0.0 1 152.3
11:47:01.852 INFO ProgressMeter - Traversal complete. Processed 1 total batches in 0.0 minutes.
11:47:01.852 INFO GenomicsDBImport - Import completed!
11:47:01.872 INFO GenomicsDBImport - Shutting down engine
[February 5, 2018 11:47:01 AM EST] org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport done. Elapsed time: 0.03 minutes.
Runtime.totalMemory()=2356150272
Tool returned:
true

Thanks

Jason

Best Answer

Answers

Sign In or Register to comment.