Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office on November 11th and 13th 2019, due to the U.S. holiday(Veteran's day) and due to a team event(Nov 13th). We will return to monitoring the GATK forum on November 12th and 14th respectively. Thank you for your patience.

CNNScoreVariants error

NicolasKNicolasK GermanyMember
Hi,

I tried to install the the gatk conda environment, which went fine, despite some minor error messages, which indicated that packages could not be installed, but they are already in ```/usr/local/lib/python3.6/dist-packages```.

Here is the gatk command after I enter the gatk ennvironment (conda activate gatk)

```(gatk) some_guy:/media/data/analysis/WES_val18_1/results/gatk$ gatk CNNScoreVariants -I ../bwa/index_dedup.bam -V index_filtered.vcf.gz -R /home/some_guy/genomes/hg38/GRCh38.primary_assembly.genome.fa -O annotated.vcf -inference-batch-size 2 -transfer-batch-size 2 -tensor-type read_tensor
Using GATK jar /home/some_guy/miniconda3/share/gatk4-4.1.2.0-1/gatk-package-4.1.2.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/some_guy/miniconda3/share/gatk4-4.1.2.0-1/gatk-package-4.1.2.0-local.jar CNNScoreVariants -I ../bwa/index_dedup.bam -V index_filtered.vcf.gz -R /home/some_guy/genomes/hg38/GRCh38.primary_assembly.genome.fa -O annotated.vcf -inference-batch-size 2 -transfer-batch-size 2 -tensor-type read_tensor
12:22:37.542 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/some_guy/miniconda3/share/gatk4-4.1.2.0-1/gatk-package-4.1.2.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
Jul 16, 2019 12:22:39 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
12:22:39.271 INFO CNNScoreVariants - ------------------------------------------------------------
12:22:39.272 INFO CNNScoreVariants - The Genome Analysis Toolkit (GATK) v4.1.2.0
12:22:39.272 INFO CNNScoreVariants - Executing as some_guy on Linux v4.15.0-43-generic amd64
12:22:39.273 INFO CNNScoreVariants - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_192-b01
12:22:39.273 INFO CNNScoreVariants - Start Date/Time: July 16, 2019 12:22:37 PM CEST
12:22:39.273 INFO CNNScoreVariants - ------------------------------------------------------------
12:22:39.273 INFO CNNScoreVariants - ------------------------------------------------------------
12:22:39.274 INFO CNNScoreVariants - HTSJDK Version: 2.19.0
12:22:39.274 INFO CNNScoreVariants - Picard Version: 2.19.0
12:22:39.274 INFO CNNScoreVariants - HTSJDK Defaults.COMPRESSION_LEVEL : 2
12:22:39.274 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
12:22:39.274 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
12:22:39.274 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
12:22:39.274 INFO CNNScoreVariants - Deflater: IntelDeflater
12:22:39.274 INFO CNNScoreVariants - Inflater: IntelInflater
12:22:39.274 INFO CNNScoreVariants - GCS max retries/reopens: 20
12:22:39.274 INFO CNNScoreVariants - Requester pays: disabled
12:22:39.275 INFO CNNScoreVariants - Initializing engine
12:22:39.658 INFO FeatureManager - Using codec VCFCodec to read file file:///media/data/analysis/WES_val18_1/results/gatk/index_filtered.vcf.gz
12:22:39.848 INFO CNNScoreVariants - Done initializing engine
12:22:39.849 INFO NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/home/some_guy/miniconda3/share/gatk4-4.1.2.0-1/gatk-package-4.1.2.0-local.jar!/com/intel/gkl/native/libgkl_utils.so
12:22:39.921 INFO CNNScoreVariants - Done scoring variants with CNN.
12:22:39.921 INFO CNNScoreVariants - Shutting down engine
[July 16, 2019 12:22:39 PM CEST] org.broadinstitute.hellbender.tools.walkers.vqsr.CNNScoreVariants done. Elapsed time: 0.04 minutes.
Runtime.totalMemory()=2210922496
java.lang.NullPointerException
at org.broadinstitute.hellbender.utils.runtime.ProcessControllerAckResult.hasMessage(ProcessControllerAckResult.java:49)
at org.broadinstitute.hellbender.utils.runtime.ProcessControllerAckResult.getDisplayMessage(ProcessControllerAckResult.java:69)
at org.broadinstitute.hellbender.utils.runtime.StreamingProcessController.waitForAck(StreamingProcessController.java:229)
at org.broadinstitute.hellbender.utils.python.StreamingPythonScriptExecutor.waitForAck(StreamingPythonScriptExecutor.java:216)
at org.broadinstitute.hellbender.utils.python.StreamingPythonScriptExecutor.sendSynchronousCommand(StreamingPythonScriptExecutor.java:183)
at org.broadinstitute.hellbender.tools.walkers.vqsr.CNNScoreVariants.onTraversalStart(CNNScoreVariants.java:307)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1037)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
at org.broadinstitute.hellbender.Main.main(Main.java:291)
```


Version:
The Genome Analysis Toolkit (GATK) v4.1.2.0
HTSJDK Version: 2.19.0
Picard Version: 2.19.0

OS:
Ubuntu 18.04 Server LTS


Kind regards,
Nicolas

Best Answers

  • NicolasKNicolasK Germany
    Accepted Answer
    For anyone who did not find the right description immediately.

    The solution was already perfectly described here:
    "software.broadinstitute.org/gatk/documentation/article?id=11090"
    For now, let's focus on starting a new instance of the GATK4 container, specifying in the following command what is your particular container ID and the filesystem location you want to mount.
    docker run -v ~/my_project:/gatk/my_data -it broadinstitute/gatk:4.1.1.0

    It is as follows for my case:
    docker run -v /media/data/analysis:/gatk/my_data -it 9e737a9f562c

    Sorry for that, hope this might help someone else.

Answers

  • NicolasKNicolasK GermanyMember
    Hi @bhanuGandham,

    thank you for your quick reply.
    yes, I followed the instructions on github.com/broadinstitute/gatk#python.
    Just got some minor errors, which did not seem to be too bad.

    With Docker I have some challenges getting my data folders mountet.
    I saw some hints in "gatkforums.broadinstitute.org/gatk/discussion/10172/how-to-run-the-gatk4-docker-locally-and-take-a-look-inside".

    Especially: " run -v, --volume=[host-src:]container-dest[:<options>]".

    Somehow that did not work for me.


    Downloading the docker image works fine.

    /media/data$ docker image ls
    REPOSITORY TAG IMAGE ID CREATED SIZE
    nginx latest 98ebf73aba75 14 hours ago 109MB
    broadinstitute/gatk latest 9e737a9f562c 2 months ago 3.84GB


    Starting the container also, but when I want to mount a device it fails:
    docker run -i -t 9e737a9f562c -v=/media/data/:/app

    I already tried several parameters, none worked.


    KInd regards,
    Nicolas
  • NicolasKNicolasK GermanyMember
    Sorry forgot the error message:
    docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"-v=/dev/sda/:/app\": stat -v=/dev/sda/:/app: no such file or directory": unknown.
  • NicolasKNicolasK GermanyMember
    Accepted Answer
    For anyone who did not find the right description immediately.

    The solution was already perfectly described here:
    "software.broadinstitute.org/gatk/documentation/article?id=11090"
    For now, let's focus on starting a new instance of the GATK4 container, specifying in the following command what is your particular container ID and the filesystem location you want to mount.
    docker run -v ~/my_project:/gatk/my_data -it broadinstitute/gatk:4.1.1.0

    It is as follows for my case:
    docker run -v /media/data/analysis:/gatk/my_data -it 9e737a9f562c

    Sorry for that, hope this might help someone else.
  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin

    Thank you @NicolasK this will help other users!

Sign In or Register to comment.