Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office on November 11th and 13th 2019, due to the U.S. holiday(Veteran's day) and due to a team event(Nov 13th). We will return to monitoring the GATK forum on November 12th and 14th respectively. Thank you for your patience.

Install with conda, gatkcondaenv.yml not found

splaisansplaisan Leuven (Belgium)Member ✭✭
edited November 2018 in Ask the GATK team

This link from https://software.broadinstitute.org/gatk/documentation/article?id=11049 is dead and no link is present in https://software.broadinstitute.org/gatk/documentation/article?id=12836.

Where can I please find gatkcondaenv.yml?

Should I use the GIT file from here ?

Thanks

Tagged:

Best Answer

Answers

  • SChaluvadiSChaluvadi Member, Broadie, Moderator admin
    edited November 2018

    @splaisan Let me double check and get back to you on the correct location of that file!

    Post edited by SChaluvadi on
  • SChaluvadiSChaluvadi Member, Broadie, Moderator admin

    @splaisan I hope that you were able to access the file! Since we have no heard back from you, we will be closing this ticket but as always, if you have any other questions feel free to respond back.

  • lakhujanivijaylakhujanivijay IndiaMember

    Go to this link

    The file link "gatkcondaenv.yml"

    under Python dependencies is broken. Kindly fix this as soon as possible.

  • alongaloralongalor Member
    edited March 6

    I'm getting an error:

    conda env create -n gatk -f gatkcondaenv.yml 
    Collecting package metadata: done
    Solving environment: done
    Preparing transaction: done
    Verifying transaction: done
    Executing transaction: done
    
    CondaEnvException: Pip subcommand failed with 
    output: 
    error: + [[ hB != hxB ]]
    + XTRACE_STATE=-x
    + [[ hxB != hxB ]]
    + VERBOSE_STATE=+v
    + set +xv
    + unset XTRACE_STATE VERBOSE_STATE
    ++ /home/ag457/anaconda3/bin/conda shell.posix hook
    + eval 'export CONDA_EXE="/home/ag457/anaconda3/bin/conda"
    # Copyright (C) 2012 Anaconda, Inc
    # SPDX-License-Identifier: BSD-3-Clause
    
    __conda_hashr() {
        if [ -n "${ZSH_VERSION:+x}" ]; then
            \rehash
        elif [ -n "${POSH_VERSION:+x}" ]; then
            :  # pass
        else
            \hash -r
        fi
    }
    
    __conda_activate() {
        if [ -n "${CONDA_PS1_BACKUP:+x}" ]; then
            # Handle transition from shell activated with conda <= 4.3 to a subsequent activation
            # after conda updated to >= 4.4. See issue #6173.
            PS1="$CONDA_PS1_BACKUP"
            \unset CONDA_PS1_BACKUP
        fi
    
        \local cmd="$1"
        shift
        \local ask_conda
        ask_conda="$(PS1="$PS1" "$CONDA_EXE" shell.posix "$cmd" "[email protected]")" || \return $?
        \eval "$ask_conda"
        __conda_hashr
    }
    
    __conda_reactivate() {
        \local ask_conda
        ask_conda="$(PS1="$PS1" "$CONDA_EXE" shell.posix reactivate)" || \return $?
        \eval "$ask_conda"
        __conda_hashr
    }
    
    conda() {
        if [ "$#" -lt 1 ]; then
            "$CONDA_EXE"
        else
            \local cmd="$1"
            shift
            case "$cmd" in
                activate|deactivate)
                    __conda_activate "$cmd" "[email protected]"
                    ;;
                install|update|upgrade|remove|uninstall)
                    "$CONDA_EXE" "$cmd" "[email protected]" && __conda_reactivate
                    ;;
                *) "$CONDA_EXE" "$cmd" "[email protected]" ;;
            esac
        fi
    }
    
    if [ -z "${CONDA_SHLVL+x}" ]; then
        \export CONDA_SHLVL=0
        PATH="$(dirname "$(dirname "$CONDA_EXE")")/condabin:${PATH:-}"
        \export PATH
    
        # We'\''re not allowing PS1 to be unbound. It must at least be set.
        # However, we'\''re not exporting it, which can cause problems when starting a second shell
        # via a first shell (i.e. starting zsh from bash).
        if [ -z "${PS1+x}" ]; then
            PS1=
        fi
    fi
    
    
    conda activate base'
    ++ export CONDA_EXE=/home/ag457/anaconda3/bin/conda
    ++ CONDA_EXE=/home/ag457/anaconda3/bin/conda
    ++ '[' -z x ']'
    ++ conda activate base
    ++ '[' 2 -lt 1 ']'
    ++ local cmd=activate
    ++ shift
    ++ case "$cmd" in
    ++ __conda_activate activate base
    ++ '[' -n '' ']'
    ++ local cmd=activate
    ++ shift
    ++ local ask_conda
    +++ PS1=
    +++ /home/ag457/anaconda3/bin/conda shell.posix activate base
    ++ ask_conda='PS1='\''(base) '\''
    \export CONDA_DEFAULT_ENV='\''base'\''
    \export CONDA_EXE='\''/home/ag457/anaconda3/bin/conda'\''
    \export CONDA_PREFIX='\''/home/ag457/anaconda3'\''
    \export CONDA_PROMPT_MODIFIER='\''(base) '\''
    \export CONDA_PYTHON_EXE='\''/home/ag457/anaconda3/bin/python'\''
    \export CONDA_SHLVL='\''1'\''
    \export PATH='\''/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/home/ag457/anaconda3/condabin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/ag457/bin:/home/ag457/bin'\''
    \. "/home/ag457/anaconda3/etc/conda/activate.d/activate-r-base.sh"
    \. "/home/ag457/anaconda3/etc/conda/activate.d/java_home.sh"'
    ++ eval 'PS1='\''(base) '\''
    \export CONDA_DEFAULT_ENV='\''base'\''
    \export CONDA_EXE='\''/home/ag457/anaconda3/bin/conda'\''
    \export CONDA_PREFIX='\''/home/ag457/anaconda3'\''
    \export CONDA_PROMPT_MODIFIER='\''(base) '\''
    \export CONDA_PYTHON_EXE='\''/home/ag457/anaconda3/bin/python'\''
    \export CONDA_SHLVL='\''1'\''
    \export PATH='\''/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/home/ag457/anaconda3/condabin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/ag457/bin:/home/ag457/bin'\''
    \. "/home/ag457/anaconda3/etc/conda/activate.d/activate-r-base.sh"
    \. "/home/ag457/anaconda3/etc/conda/activate.d/java_home.sh"'
    +++ PS1='(base) '
    +++ export CONDA_DEFAULT_ENV=base
    +++ CONDA_DEFAULT_ENV=base
    +++ export CONDA_EXE=/home/ag457/anaconda3/bin/conda
    +++ CONDA_EXE=/home/ag457/anaconda3/bin/conda
    +++ export CONDA_PREFIX=/home/ag457/anaconda3
    +++ CONDA_PREFIX=/home/ag457/anaconda3
    +++ export 'CONDA_PROMPT_MODIFIER=(base) '
    +++ CONDA_PROMPT_MODIFIER='(base) '
    +++ export CONDA_PYTHON_EXE=/home/ag457/anaconda3/bin/python
    +++ CONDA_PYTHON_EXE=/home/ag457/anaconda3/bin/python
    +++ export CONDA_SHLVL=1
    +++ CONDA_SHLVL=1
    +++ export PATH=/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/home/ag457/anaconda3/condabin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/ag457/bin:/home/ag457/bin
    +++ PATH=/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/home/ag457/anaconda3/condabin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/ag457/bin:/home/ag457/bin
    +++ . /home/ag457/anaconda3/etc/conda/activate.d/activate-r-base.sh
    ++++ R CMD javareconf
    +++ . /home/ag457/anaconda3/etc/conda/activate.d/java_home.sh
    ++++ export JAVA_HOME_CONDA_BACKUP=
    ++++ JAVA_HOME_CONDA_BACKUP=
    ++++ export JAVA_HOME=/home/ag457/anaconda3
    ++++ JAVA_HOME=/home/ag457/anaconda3
    ++ __conda_hashr
    ++ '[' -n '' ']'
    ++ '[' -n '' ']'
    ++ hash -r
    + conda activate /home/ag457/anaconda3/envs/gatk
    + '[' 2 -lt 1 ']'
    + local cmd=activate
    + shift
    + case "$cmd" in
    + __conda_activate activate /home/ag457/anaconda3/envs/gatk
    + '[' -n '' ']'
    + local cmd=activate
    + shift
    + local ask_conda
    ++ PS1='(base) '
    ++ /home/ag457/anaconda3/bin/conda shell.posix activate /home/ag457/anaconda3/envs/gatk
    + ask_conda='\. "/home/ag457/anaconda3/etc/conda/deactivate.d/java_home.sh"
    PS1='\''(gatk) '\''
    \export CONDA_DEFAULT_ENV='\''gatk'\''
    \export CONDA_PREFIX='\''/home/ag457/anaconda3/envs/gatk'\''
    \export CONDA_PREFIX_1='\''/home/ag457/anaconda3'\''
    \export CONDA_PROMPT_MODIFIER='\''(gatk) '\''
    \export CONDA_SHLVL='\''2'\''
    \export PATH='\''/home/ag457/anaconda3/envs/gatk/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/home/ag457/anaconda3/condabin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/ag457/bin:/home/ag457/bin'\'''
    + eval '\. "/home/ag457/anaconda3/etc/conda/deactivate.d/java_home.sh"
    PS1='\''(gatk) '\''
    \export CONDA_DEFAULT_ENV='\''gatk'\''
    \export CONDA_PREFIX='\''/home/ag457/anaconda3/envs/gatk'\''
    \export CONDA_PREFIX_1='\''/home/ag457/anaconda3'\''
    \export CONDA_PROMPT_MODIFIER='\''(gatk) '\''
    \export CONDA_SHLVL='\''2'\''
    \export PATH='\''/home/ag457/anaconda3/envs/gatk/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/home/ag457/anaconda3/condabin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/ag457/bin:/home/ag457/bin'\'''
    ++ . /home/ag457/anaconda3/etc/conda/deactivate.d/java_home.sh
    +++ export JAVA_HOME=
    +++ JAVA_HOME=
    ++ PS1='(gatk) '
    ++ export CONDA_DEFAULT_ENV=gatk
    ++ CONDA_DEFAULT_ENV=gatk
    ++ export CONDA_PREFIX=/home/ag457/anaconda3/envs/gatk
    ++ CONDA_PREFIX=/home/ag457/anaconda3/envs/gatk
    ++ export CONDA_PREFIX_1=/home/ag457/anaconda3
    ++ CONDA_PREFIX_1=/home/ag457/anaconda3
    ++ export 'CONDA_PROMPT_MODIFIER=(gatk) '
    ++ CONDA_PROMPT_MODIFIER='(gatk) '
    ++ export CONDA_SHLVL=2
    ++ CONDA_SHLVL=2
    ++ export PATH=/home/ag457/anaconda3/envs/gatk/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/home/ag457/anaconda3/condabin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/ag457/bin:/home/ag457/bin
    ++ PATH=/home/ag457/anaconda3/envs/gatk/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/home/ag457/anaconda3/condabin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/home/ag457/anaconda3/bin:/n/cluster/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/home/ag457/bin:/home/ag457/bin
    + __conda_hashr
    + '[' -n '' ']'
    + '[' -n '' ']'
    + hash -r
    + pip install -r /n/data1/hms/dbmi/park/alon/DATA/LowCoverage_Glia_03-04-2019/condaenv.aq9g6rz8.requirements.txt
    Invalid requirement: '$tensorFlowDependency'
    Traceback (most recent call last):
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 92, in __init__
        req = REQUIREMENT.parseString(requirement_string)
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1617, in parseString
        raise exc
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1607, in parseString
        loc, tokens = self._parse( instring, 0 )
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
        loc,tokens = self.parseImpl( instring, preloc, doActions )
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3376, in parseImpl
        loc, exprtokens = e._parse( instring, loc, doActions )
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
        loc,tokens = self.parseImpl( instring, preloc, doActions )
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3698, in parseImpl
        return self.expr._parse( instring, loc, doActions, callPreParse=False )
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1379, in _parseNoCache
        loc,tokens = self.parseImpl( instring, preloc, doActions )
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 3359, in parseImpl
        loc, resultlist = self.exprs[0]._parse( instring, loc, doActions, callPreParse=False )
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 1383, in _parseNoCache
        loc,tokens = self.parseImpl( instring, preloc, doActions )
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/pyparsing.py", line 2670, in parseImpl
        raise ParseException(instring, loc, self.errmsg, self)
    pip._vendor.pyparsing.ParseException: Expected W:(abcd...) (at char 0), (line:1, col:1)
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/req/req_install.py", line 82, in __init__
        req = Requirement(req)
      File "/home/ag457/anaconda3/envs/gatk/lib/python3.6/site-packages/pip/_vendor/packaging/requirements.py", line 96, in __init__
        requirement_string[e.loc:e.loc + 8]))
    pip._vendor.packaging.requirements.InvalidRequirement: Invalid requirement, parse error at "'$tensorF'"
    
    You are using pip version 9.0.1, however version 19.0.3 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    

    Looks like this might be caused by the line - $tensorFlowDependency. Will remove and see what happens

  • alongaloralongalor Member

    Next error is: FileNotFoundError: [Errno 2] No such file or directory: '/n/data1/hms/dbmi/park/alon/DATA/LowCoverage_Glia_03-04-2019/gatkPythonPackageArchive.zip'

  • sleeslee Member, Broadie, Dev ✭✭✭
    edited March 6

    @alongalor please see the instructions for installing the conda environment in the "Python dependencies" section at: https://github.com/broadinstitute/gatk/blob/master/README.md#requirements

    The file gatkcondaenv.yml.template is simply a template for creating the gatkcondaenv.yml file that you should be using in the conda env create -n gatk -f gatkcondaenv.yml coommand. Various things (like the - $tensorFlowDependency line that you removed and the location of gatkPythonPackageArchive.zip) are filled in appropriately, either 1) in the gatkcondaenv.yml file that is distributed with zip/tar releases or 2) when running ./gradlew localDevCondaEnv from a git clone of the repository, as described in the README.

  • lakhujanivijaylakhujanivijay IndiaMember

    Hi @SChaluvadi

    Where can I find the gatkcondaenv.yml file for the installation of GATK 4.1.2.0 ? I could not find it on the github account. Can you please help me out?

  • bshifawbshifaw Member, Broadie, Moderator admin

    lakhujanivijay question is a duplicate and has been answered here

Sign In or Register to comment.