We've moved!
This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!

Errors when I run the BaseRecalibrator

wulywuly Member
edited May 2019 in Ask the GATK team

Hi,I got some errors when I used BaseRecalibrator. It can run at first ,but it got some errors after a while.
The command is :

/data/home/wuly/soft/GATK4/gatk-4.1.2.0/gatk --java-options "-Xmx20G -Djava.io.tmpdir=./" BaseRecalibrator \
-R /data/home/wuly/source/Homo_sapiens_assembly38.fasta \
-I /data/home/wuly/breast_cancer/clean_data/M1/markup/CL100066321_L02_547.sorted.marked.bam \
--known-sites /data/home/wuly/source/hapmap_3.3.hg38.vcf.gz \
--known-sites /data/home/wuly/source/dbsnp_146.hg38.vcf.gz \
--known-sites /data/home/wuly/source/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz \
--known-sites /data/home/wuly/source/1000G_phase1.snps.high_confidence.hg38.vcf.gz \
-O M1_recal.table

This is the log:

Using GATK jar /data/home/wuly/soft/GATK4/gatk-4.1.2.0/gatk-package-4.1.2.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx20G -Djava.io.tmpdir=./
 -jar /data/home/wuly/soft/GATK4/gatk-4.1.2.0/gatk-package-4.1.2.0-local.jar BaseRecalibrator -R /data/home/wuly/source/Homo_sapiens_assembly38.fasta -I /data/home/wuly/breast_cancer/clean_data/M1.bam --known-sites /data/home/wuly/source/hapmap_3.3.hg38.vcf.gz --known-sites /data/home/wuly/source/dbsnp_146.hg38.vcf.gz --known-sites /data/home/wuly/source/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz --known-sites /data/home/wuly/source/1000G_phase1.snps.high_confidence.hg38.vcf.gz -O M1_recal.table23:26:23.393 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/data/home/wuly/soft/GATK4/gatk-4.1.2.0/gatk-package-4.1.2.0-local.jar!/com/intel/gkl/native/libgkl_compre
ssion.soMay 28, 2019 11:26:25 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
23:26:25.147 INFO  BaseRecalibrator - ------------------------------------------------------------
23:26:25.147 INFO  BaseRecalibrator - The Genome Analysis Toolkit (GATK) v4.1.2.0
23:26:25.147 INFO  BaseRecalibrator - For support and documentation go to https://software.broadinstitute.org/gatk/
23:26:25.147 INFO  BaseRecalibrator - Executing as [email protected] on Linux v3.10.0-957.10.1.el7.x86_64 amd64
23:26:25.147 INFO  BaseRecalibrator - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_192-b01
23:26:25.148 INFO  BaseRecalibrator - Start Date/Time: May 28, 2019 11:26:23 PM EDT
23:26:25.148 INFO  BaseRecalibrator - ------------------------------------------------------------
23:26:25.148 INFO  BaseRecalibrator - ------------------------------------------------------------
23:26:25.148 INFO  BaseRecalibrator - HTSJDK Version: 2.19.0
23:26:25.148 INFO  BaseRecalibrator - Picard Version: 2.19.0
23:26:25.148 INFO  BaseRecalibrator - HTSJDK Defaults.COMPRESSION_LEVEL : 2
23:26:25.148 INFO  BaseRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
23:26:25.148 INFO  BaseRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
23:26:25.148 INFO  BaseRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
23:26:25.148 INFO  BaseRecalibrator - Deflater: IntelDeflater
23:26:25.148 INFO  BaseRecalibrator - Inflater: IntelInflater
23:26:25.148 INFO  BaseRecalibrator - GCS max retries/reopens: 20
23:26:25.148 INFO  BaseRecalibrator - Requester pays: disabled
23:26:25.148 INFO  BaseRecalibrator - Initializing engine
23:26:25.686 INFO  FeatureManager - Using codec VCFCodec to read file file:///data/home/wuly/source/hapmap_3.3.hg38.vcf.gz
23:26:25.821 INFO  FeatureManager - Using codec VCFCodec to read file file:///data/home/wuly/source/dbsnp_146.hg38.vcf.gz
23:26:25.906 INFO  FeatureManager - Using codec VCFCodec to read file file:///data/home/wuly/source/Mills_and_1000G_gold_standard.indels.hg38.vcf.gz
23:26:25.984 INFO  FeatureManager - Using codec VCFCodec to read file file:///data/home/wuly/source/1000G_phase1.snps.high_confidence.hg38.vcf.gz
23:26:26.082 WARN  IndexUtils - Feature file "/data/home/wuly/source/dbsnp_146.hg38.vcf.gz" appears to contain no sequence dictionary. Attempting to retrieve a sequence dictionary from the as
sociated index file23:26:26.156 WARN  IndexUtils - Index file /data/home/wuly/source/dbsnp_146.hg38.vcf.gz.tbi is out of date (index older than input file). Use IndexFeatureFile to make a new index.
23:26:26.221 INFO  BaseRecalibrator - Done initializing engine
23:26:26.225 INFO  BaseRecalibrationEngine - The covariates being used here: 
23:26:26.225 INFO  BaseRecalibrationEngine -    ReadGroupCovariate
23:26:26.225 INFO  BaseRecalibrationEngine -    QualityScoreCovariate
23:26:26.225 INFO  BaseRecalibrationEngine -    ContextCovariate
23:26:26.225 INFO  BaseRecalibrationEngine -    CycleCovariate
23:26:26.298 INFO  ProgressMeter - Starting traversal
23:26:26.299 INFO  ProgressMeter -        Current Locus  Elapsed Minutes       Reads Processed     Reads/Minute
23:26:36.311 INFO  ProgressMeter -         chr1:3022190              0.2                292000        1750074.9
23:26:46.314 INFO  ProgressMeter -         chr1:6578963              0.3                495000        1483961.2
23:26:56.318 INFO  ProgressMeter -         chr1:9975774              0.5                691000        1381171.3
23:27:06.372 INFO  ProgressMeter -        chr1:12234125              0.7                953000        1426931.5
23:27:16.382 INFO  ProgressMeter -        chr1:15383910              0.8               1139000        1364562.1
23:27:26.420 INFO  ProgressMeter -        chr1:17587500              1.0               1452000        1449101.8
23:27:36.446 INFO  ProgressMeter -        chr1:20483560              1.2               1658000        1418164.7
....
23:41:58.683 INFO  BaseRecalibrator - Shutting down engine
[May 28, 2019 11:41:58 PM EDT] org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator done. Elapsed time: 15.59 minutes.
Runtime.totalMemory()=4510973952
htsjdk.samtools.SAMFormatException: Did not inflate expected amount
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:147)
    at htsjdk.samtools.util.BlockGunzipper.unzipBlock(BlockGunzipper.java:96)
    at htsjdk.samtools.util.BlockCompressedInputStream.inflateBlock(BlockCompressedInputStream.java:550)
    at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:532)
    at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
    at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:458)
    at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:196)
    at htsjdk.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:241)
    at htsjdk.tribble.readers.TabixReader.readLine(TabixReader.java:215)
    at htsjdk.tribble.readers.TabixReader.access$300(TabixReader.java:48)
    at htsjdk.tribble.readers.TabixReader$IteratorImpl.next(TabixReader.java:434)
    at htsjdk.tribble.readers.TabixIteratorLineReader.readLine(TabixIteratorLineReader.java:46)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.readNextRecord(TabixFeatureReader.java:170)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.next(TabixFeatureReader.java:205)
    at htsjdk.tribble.TabixFeatureReader$FeatureIterator.next(TabixFeatureReader.java:149)
    at org.broadinstitute.hellbender.engine.FeatureCache.fill(FeatureCache.java:141)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:534)
    at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:502)
    at org.broadinstitute.hellbender.engine.FeatureManager.getFeatures(FeatureManager.java:340)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:172)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:124)
    at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:262)
    at org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator.apply(BaseRecalibrator.java:185)
    at org.broadinstitute.hellbender.engine.ReadWalker.lambda$traverse$0(ReadWalker.java:96)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.Iterator.forEachRemaining(Iterator.java:116)
    at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    at org.broadinstitute.hellbender.engine.ReadWalker.traverse(ReadWalker.java:94)
    at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1039)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
    at org.broadinstitute.hellbender.Main.main(Main.java:291)

I also run the ValidateSamFile, but it reported:no error found.

Post edited by bshifaw on

Best Answers

Answers

  • wulywuly Member
    edited May 2019

    Thanks!@bshifaw ,
    I checked my disk space :

    (base) [[email protected] ~]$ df -lh
    Filesystem               Size  Used Avail Use% Mounted on
    /dev/mapper/centos-root   50G  6.3G   44G  13% /
    devtmpfs                 472G     0  472G   0% /dev
    tmpfs                    473G     0  473G   0% /dev/shm
    tmpfs                    473G   37M  472G   1% /run
    tmpfs                    473G     0  473G   0% /sys/fs/cgroup
    /dev/sda2               1014M  234M  781M  23% /boot
    /dev/mapper/centos-home  2.2T   37M  2.2T   1% /home
    tmpfs                     95G   12K   95G   1% /run/user/42
    /dev/mapper/vg1-lv1      8.7T  1.8T  6.5T  22% /data
    tmpfs                     95G     0   95G   0% /run/user/1003
    

    I think that there is still a lot of disk space left .I also tried to change the path for "-Djava.io.tmpdir=" ,but it still can't work.

    Post edited by bshifaw on
Sign In or Register to comment.