problem of calling SNPs GenotypeGVCFs

Dear Mr/Ms.,

I used GenomicsDBImport and GenotypeGVCFs to call SNPs in GATK4. Due to large reference genome (1G) and sample size (100). I want to separate the work for each chromosome. It seems that GenomicsDBImport works for all chromosomes, but GenotypeGVCFs only works for chromosome1. Could you please give me some suggestions. Below are commands and log information for chromosome 2.

Look forward to hearing from you soon.
Best regards,
Baosheng

command, all variables are defined before command lines.

$GATK --java-options "-Xmx24g" \
GenomicsDBImport \
${InputVCF} \
--genomicsdb-workspace-path ${OUTDIR}/chr02 \
-L Qrob_Chr02

$GATK --java-options "-Xmx48g" \
GenotypeGVCFs \
-R ${REF} \
-V gendb://${OUTDIR}/chr02 \
-all-sites \
-O ${OUTDIR}/chr02.vcf

log file

23:41:40.274 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/WangBS/software/GATK/gatk/build/libs/gatk-package-4.0.11.0-56-g2c0e9b0-SNAPSHOT-local.jar!/com/intel/gkl/native/libgkl_compression.so
23:42:41.990 INFO GenomicsDBImport - ------------------------------------------------------------
23:42:41.990 INFO GenomicsDBImport - The Genome Analysis Toolkit (GATK) v4.0.11.0-56-g2c0e9b0-SNAPSHOT
23:42:41.990 INFO GenomicsDBImport - For support and documentation go to https://software.broadinstitute.org/gatk/
23:42:41.991 INFO GenomicsDBImport - Executing as [email protected] on Linux v3.10.0-693.el7.x86_64 amd64
23:42:41.991 INFO GenomicsDBImport - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_131-b12
23:42:41.991 INFO GenomicsDBImport - Start Date/Time: January 26, 2019 11:41:40 PM CST
23:42:41.991 INFO GenomicsDBImport - ------------------------------------------------------------
23:42:41.991 INFO GenomicsDBImport - ------------------------------------------------------------
23:42:41.991 INFO GenomicsDBImport - HTSJDK Version: 2.18.1
23:42:41.991 INFO GenomicsDBImport - Picard Version: 2.18.16
23:42:41.992 INFO GenomicsDBImport - HTSJDK Defaults.COMPRESSION_LEVEL : 2
23:42:41.992 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
23:42:41.992 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
23:42:41.992 INFO GenomicsDBImport - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
23:42:41.992 INFO GenomicsDBImport - Deflater: IntelDeflater
23:42:41.992 INFO GenomicsDBImport - Inflater: IntelInflater
23:42:41.992 INFO GenomicsDBImport - GCS max retries/reopens: 20
23:42:41.992 INFO GenomicsDBImport - Requester pays: disabled
23:42:41.992 INFO GenomicsDBImport - Initializing engine
23:42:44.099 INFO IntervalArgumentCollection - Processing 115639695 bp from intervals
23:42:44.102 INFO GenomicsDBImport - Done initializing engine
23:42:44.276 INFO GenomicsDBImport - Vid Map JSON file will be written to /home/WangBS/Analyses/vcf/test/chr02/vidmap.json
23:42:44.276 INFO GenomicsDBImport - Callset Map JSON file will be written to /home/WangBS/Analyses/vcf/test/chr02/callset.json
23:42:44.276 INFO GenomicsDBImport - Complete VCF Header will be written to /home/WangBS/Analyses/vcf/test/chr02/vcfheader.vcf
23:42:44.276 INFO GenomicsDBImport - Importing to array - /home/WangBS/Analyses/vcf/test/chr02/genomicsdb_array
23:42:44.276 INFO ProgressMeter - Starting traversal
23:42:44.276 INFO ProgressMeter - Current Locus Elapsed Minutes Batches Processed Batches/Minute
23:42:45.830 INFO GenomicsDBImport - Importing batch 1 with 63 samples
Buffer resized from 37294 bytes to 65464
Buffer resized from 37294 bytes to 65511
Buffer resized from 37293 bytes to 65539
Buffer resized from 37294 bytes to 65447
.....
.....
Buffer resized from 65538 bytes to 65539
Buffer resized from 65538 bytes to 65539
Buffer resized from 65538 bytes to 65539
06:50:14.219 INFO ProgressMeter - Qrob_Chr02:1 427.5 1 0.0
06:50:14.220 INFO GenomicsDBImport - Done importing batch 1/1
06:50:14.221 INFO ProgressMeter - Qrob_Chr02:1 427.5 1 0.0
06:50:14.229 INFO ProgressMeter - Traversal complete. Processed 1 total batches in 427.5 minutes.
06:50:14.236 INFO GenomicsDBImport - Import completed!
06:50:14.236 INFO GenomicsDBImport - Shutting down engine
[January 27, 2019 6:50:14 AM CST] org.broadinstitute.hellbender.tools.genomicsdb.GenomicsDBImport done. Elapsed time: 428.57 minutes.
Runtime.totalMemory()=8988393472
Tool returned:
true
Using GATK jar /home/WangBS/software/GATK/gatk/build/libs/gatk-package-4.0.11.0-56-g2c0e9b0-SNAPSHOT-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx24g -jar /home/WangBS/software/GATK/gatk/build/libs/gatk-package-4.0.11.0-56-g2c0e9b0-SNAPSHOT-local.jar GenotypeGVCFs -R /home/WangBS/Reference/Qrobur/Qrob_PM1N.fa -V gendb:///home/WangBS/Analyses/vcf/test/chr02 -all-sites -O /home/WangBS/Analyses/vcf/test/chr02.vcf
06:50:19.236 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/WangBS/software/GATK/gatk/build/libs/gatk-package-4.0.11.0-56-g2c0e9b0-SNAPSHOT-local.jar!/com/intel/gkl/native/libgkl_compression.so
06:51:21.116 INFO GenotypeGVCFs - ------------------------------------------------------------
06:51:21.116 INFO GenotypeGVCFs - The Genome Analysis Toolkit (GATK) v4.0.11.0-56-g2c0e9b0-SNAPSHOT
06:51:21.116 INFO GenotypeGVCFs - For support and documentation go to https://software.broadinstitute.org/gatk/
06:51:21.117 INFO GenotypeGVCFs - Executing as [email protected] on Linux v3.10.0-693.el7.x86_64 amd64
06:51:21.117 INFO GenotypeGVCFs - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_131-b12
06:51:21.117 INFO GenotypeGVCFs - Start Date/Time: January 27, 2019 6:50:19 AM CST
06:51:21.117 INFO GenotypeGVCFs - ------------------------------------------------------------
06:51:21.117 INFO GenotypeGVCFs - ------------------------------------------------------------
06:51:21.118 INFO GenotypeGVCFs - HTSJDK Version: 2.18.1
06:51:21.118 INFO GenotypeGVCFs - Picard Version: 2.18.16
06:51:21.118 INFO GenotypeGVCFs - HTSJDK Defaults.COMPRESSION_LEVEL : 2
06:51:21.118 INFO GenotypeGVCFs - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
06:51:21.118 INFO GenotypeGVCFs - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
06:51:21.118 INFO GenotypeGVCFs - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
06:51:21.118 INFO GenotypeGVCFs - Deflater: IntelDeflater
06:51:21.118 INFO GenotypeGVCFs - Inflater: IntelInflater
06:51:21.118 INFO GenotypeGVCFs - GCS max retries/reopens: 20
06:51:21.118 INFO GenotypeGVCFs - Requester pays: disabled
06:51:21.118 INFO GenotypeGVCFs - Initializing engine
WARNING: No valid combination operation found for INFO field DS - the field will NOT be part of INFO fields in the generated VCF records
WARNING: No valid combination operation found for INFO field InbreedingCoeff - the field will NOT be part of INFO fields in the generated VCF records
WARNING: No valid combination operation found for INFO field MLEAC - the field will NOT be part of INFO fields in the generated VCF records
WARNING: No valid combination operation found for INFO field MLEAF - the field will NOT be part of INFO fields in the generated VCF records
WARNING: No valid combination operation found for INFO field DS - the field will NOT be part of INFO fields in the generated VCF records
WARNING: No valid combination operation found for INFO field InbreedingCoeff - the field will NOT be part of INFO fields in the generated VCF records
WARNING: No valid combination operation found for INFO field MLEAC - the field will NOT be part of INFO fields in the generated VCF records
WARNING: No valid combination operation found for INFO field MLEAF - the field will NOT be part of INFO fields in the generated VCF records
06:51:26.212 INFO GenotypeGVCFs - Done initializing engine
06:51:26.257 INFO ProgressMeter - Starting traversal
06:51:26.257 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute
06:51:26.278 INFO GenotypeGVCFs - Shutting down engine
[January 27, 2019 6:51:26 AM CST] org.broadinstitute.hellbender.tools.walkers.GenotypeGVCFs done. Elapsed time: 1.12 minutes.
Runtime.totalMemory()=1972371456
java.lang.IllegalStateException: There are no sources based on those query parameters
at com.intel.genomicsdb.reader.GenomicsDBFeatureIterator.(GenomicsDBFeatureIterator.java:131)
at com.intel.genomicsdb.reader.GenomicsDBFeatureReader.query(GenomicsDBFeatureReader.java:144)
at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:534)
at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:503)
at org.broadinstitute.hellbender.engine.FeatureDataSource.query(FeatureDataSource.java:469)
at org.broadinstitute.hellbender.engine.VariantLocusWalker.lambda$traverse$2(VariantLocusWalker.java:144)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.ReferencePipeline$Head.forEachOrdered(ReferencePipeline.java:590)
at org.broadinstitute.hellbender.engine.VariantLocusWalker.traverse(VariantLocusWalker.java:143)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:966)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
at org.broadinstitute.hellbender.Main.main(Main.java:289)

Issue · Github
by bhanuGandham

Issue Number
5865
State
open
Last Updated
Assignee
Array

Best Answer

Answers

Sign In or Register to comment.