To celebrate the release of GATK 4.0, we are giving away free credits for running the GATK4 Best Practices pipelines in FireCloud, our secure online analysis portal. It’s first come first serve, so sign up now to claim your free credits worth $250. Sponsored by Google Cloud. Learn more at https://software.broadinstitute.org/firecloud/documentation/freecredits

GetBayesianHetCoverage exits with code 247

When I run GetBayesianHetCoverage in a Docker container, the function exits without completing with code 247. The Docker container is built from an image that uses the Debian "Stretch" base image, with openjdk version "1.8.0_141" and GATK4.beta.5 added. When I run the same code on my local machine with the same OS, java, and GATK versions, the code runs fine. However, whenever the code is run inside my Docker container, whether locally or on a cloud service, I see this error. Any ideas? Error messages are below.

Traceback (most recent call last):
File "call_het_snps_and_merge_coverage_workflow_launch.py", line 83, in
main()
File "call_het_snps_and_merge_coverage_workflow_launch.py", line 69, in main
gatk_tmp_dir=paths['temp'])
File "/usr/gitc/call_het_snps_and_merge_coverage_workflow.py", line 312, in call_het_snps_and_merge_coverage_workflow
get_het_step()
File "/usr/local/lib/python3.5/dist-packages/plumbum/commands/base.py", line 96, in call
return self.run(args, **kwargs)[1]
File "/usr/local/lib/python3.5/dist-packages/plumbum/commands/base.py", line 226, in run
return p.run()
File "/usr/local/lib/python3.5/dist-packages/plumbum/commands/base.py", line 188, in runner
return run_proc(p, retcode, timeout)
File "/usr/local/lib/python3.5/dist-packages/plumbum/commands/processes.py", line 206, in run_proc
return _check_process(proc, retcode, timeout, stdout, stderr)
File "/usr/local/lib/python3.5/dist-packages/plumbum/commands/processes.py", line 23, in _check_process
proc.verify(retcode, timeout, stdout, stderr)
File "/usr/local/lib/python3.5/dist-packages/plumbum/machines/base.py", line 22, in verify
stdout, stderr)
plumbum.commands.processes.ProcessExecutionError: Command line: ['/usr/gitc/gatk-4.beta.5/gatk-launch', '--javaOptions', '-Xmx32g -Djava.io.tmpdir=/docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/temp', 'GetBayesianHetCoverage', '--reference', '/docker_scratch/reference/Homo_sapiens_assembly38.fasta', '--snpIntervals', '/docker_scratch/reference/1000G.phase1.omni2.5.hapmap3.3.ENTIRE.dbsnp146.PADDEDINTERVALSONLY.interval_list', '--tumor', '/docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/aligned/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.bam', '--tumorHets', '/docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/het_coverage/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.het.tsv', '--minimumAbnormalFraction', '0.5', '--maximumAbnormalFraction', '0.8', '--readDepthThreshold', '15', '--quadratureOrder', '200', '--hetCallingStringency', '30.0']
Exit code: 247
Stdout: | 15:00:49.096 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/usr/gitc/gatk-4.beta.5/gatk-package-4.beta.5-local.jar!/com/intel/gkl/native/libgkl_compression.so
| 15:00:49.275 INFO GetBayesianHetCoverage - HTSJDK Defaults.COMPRESSION_LEVEL : 1
| 15:00:49.275 INFO GetBayesianHetCoverage - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
| 15:00:49.275 INFO GetBayesianHetCoverage - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
| 15:00:49.275 INFO GetBayesianHetCoverage - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
| 15:00:49.275 INFO GetBayesianHetCoverage - Deflater: IntelDeflater
| 15:00:49.275 INFO GetBayesianHetCoverage - Inflater: IntelInflater
| 15:00:49.275 INFO GetBayesianHetCoverage - GCS max retries/reopens: 20
| 15:00:49.275 INFO GetBayesianHetCoverage - Using google-cloud-java patch c035098b5e62cb4fe9155eff07ce88449a361f5d from https://github.com/droazen/google-cloud-java/tree/dr_all_nio_fixes
| 15:00:49.275 INFO GetBayesianHetCoverage - Initializing engine
| 15:00:49.275 INFO GetBayesianHetCoverage - Done initializing engine
| 15:00:49.275 INFO GetBayesianHetCoverage - TUMOR_ONLY mode selected.
Stderr: | Using GATK jar /usr/gitc/gatk-4.beta.5/gatk-package-4.beta.5-local.jar
| Running:
| java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -Dsnappy.disable=true -Xmx32g -Djava.io.tmpdir=/docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/temp -jar /usr/gitc/gatk-4.beta.5/gatk-package-4.beta.5-local.jar GetBayesianHetCoverage --reference /docker_scratch/reference/Homo_sapiens_assembly38.fasta --snpIntervals /docker_scratch/reference/1000G.phase1.omni2.5.hapmap3.3.ENTIRE.dbsnp146.PADDEDINTERVALSONLY.interval_list --tumor /docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/aligned/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.bam --tumorHets /docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/het_coverage/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.het.tsv --minimumAbnormalFraction 0.5 --maximumAbnormalFraction 0.8 --readDepthThreshold 15 --quadratureOrder 200 --hetCallingStringency 30.0
| [September 27, 2017 3:00:49 PM UTC] GetBayesianHetCoverage --reference /docker_scratch/reference/Homo_sapiens_assembly38.fasta --tumor /docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/aligned/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.bam --snpIntervals /docker_scratch/reference/1000G.phase1.omni2.5.hapmap3.3.ENTIRE.dbsnp146.PADDEDINTERVALSONLY.interval_list --tumorHets /docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/het_coverage/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.het.tsv --readDepthThreshold 15 --hetCallingStringency 30.0 --minimumAbnormalFraction 0.5 --maximumAbnormalFraction 0.8 --quadratureOrder 200 --minimumMappingQuality 30 --minimumBaseQuality 20 --VALIDATION_STRINGENCY SILENT --maximumCopyNumber 2 --errorAdjustmentFactor 1.0 --help false --version false --showHidden false --verbosity INFO --QUIET false --use_jdk_deflater false --use_jdk_inflater false --gcs_max_retries 20
| [September 27, 2017 3:00:49 PM UTC] Executing as root@7b36c74459e6 on Linux 4.9.41-moby amd64; OpenJDK 64-Bit Server VM 1.8.0_141-8u141-b15-1~deb9u1-b15; Version: 4.beta.5

Tagged:

Issue · Github
by Sheila

Issue Number
2551
State
closed
Last Updated
Assignee
Array
Milestone
Array
Closed By
chandrans

Answers

  • Further comments: I have now tried a different version of GATK (4.beta.3) and Oracle JDK (1.8.0_144) in the Docker container on my local machine and see the same error. When the call to GetBayesianHetCoverage is run on its own at the command line (without the enclosing function) it produces the output pasted below, runs for two minutes, and then exits without writing a .het.tsv output file.

    Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=1 -Dsnappy.disable=true -Xmx32g -Djava.io.tmpdir=/docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/temp -jar /usr/gitc/gatk-4.beta.5/gatk-package-4.beta.5-local.jar GetBayesianHetCoverage --reference /docker_scratch/reference/Homo_sapiens_assembly38.fasta --snpIntervals /docker_scratch/reference/1000G.phase1.omni2.5.hapmap3.3.ENTIRE.dbsnp146.PADDEDINTERVALSONLY.interval_list --tumor /docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/aligned/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.bam --tumorHets /docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/het_coverage/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.het.tsv --minimumAbnormalFraction 0.5 --maximumAbnormalFraction 0.8 --readDepthThreshold 15 --quadratureOrder 200 --hetCallingStringency 30.0
    17:31:29.547 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/usr/gitc/gatk-4.beta.5/gatk-package-4.beta.5-local.jar!/com/intel/gkl/native/libgkl_compression.so
    [September 27, 2017 5:31:29 PM UTC] GetBayesianHetCoverage --reference /docker_scratch/reference/Homo_sapiens_assembly38.fasta --tumor /docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/aligned/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.bam --snpIntervals /docker_scratch/reference/1000G.phase1.omni2.5.hapmap3.3.ENTIRE.dbsnp146.PADDEDINTERVALSONLY.interval_list --tumorHets /docker_scratch/gSEQWorkflow-b6e4b3ee-d898-4534-92c5-598269d78433/het_coverage/0_S2_L001_001_170829_M05411_0012_000000000-BDYV8.het.tsv --readDepthThreshold 15 --hetCallingStringency 30.0 --minimumAbnormalFraction 0.5 --maximumAbnormalFraction 0.8 --quadratureOrder 200 --minimumMappingQuality 30 --minimumBaseQuality 20 --VALIDATION_STRINGENCY SILENT --maximumCopyNumber 2 --errorAdjustmentFactor 1.0 --help false --version false --showHidden false --verbosity INFO --QUIET false --use_jdk_deflater false --use_jdk_inflater false --gcs_max_retries 20
    [September 27, 2017 5:31:29 PM UTC] Executing as root@7b36c74459e6 on Linux 4.9.41-moby amd64; OpenJDK 64-Bit Server VM 1.8.0_141-8u141-b15-1~deb9u1-b15; Version: 4.beta.5
    17:32:06.945 INFO GetBayesianHetCoverage - HTSJDK Defaults.COMPRESSION_LEVEL : 1
    17:32:06.946 INFO GetBayesianHetCoverage - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
    17:32:06.947 INFO GetBayesianHetCoverage - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
    17:32:06.948 INFO GetBayesianHetCoverage - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
    17:32:06.949 INFO GetBayesianHetCoverage - Deflater: IntelDeflater
    17:32:06.950 INFO GetBayesianHetCoverage - Inflater: IntelInflater
    17:32:06.950 INFO GetBayesianHetCoverage - GCS max retries/reopens: 20
    17:32:06.950 INFO GetBayesianHetCoverage - Using google-cloud-java patch c035098b5e62cb4fe9155eff07ce88449a361f5d from https://github.com/droazen/google-cloud-java/tree/dr_all_nio_fixes
    17:32:06.951 INFO GetBayesianHetCoverage - Initializing engine
    17:32:06.951 INFO GetBayesianHetCoverage - Done initializing engine
    17:32:06.951 INFO GetBayesianHetCoverage - TUMOR_ONLY mode selected.

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @mowreyw
    Hi,

    Let me check with the team and get back to you.

    -Sheila

  • SheilaSheila Broad InstituteMember, Broadie, Moderator

    @mowreyw
    Hi again,

    I have not forgotten about you. Sorry for the delay. I will get back to you asap, once I have an answer from the team. Feel free to post again if I don't respond by mid-week.

    Thanks,
    Sheila

  • SheilaSheila Broad InstituteMember, Broadie, Moderator
    edited November 2017

    @mowreyw
    Hi,

    This tool is being deprecated, so the best thing to do is use GetHetCoverage.

    -Sheila

Sign In or Register to comment.