The current GATK version is 3.6-0
Examples: Monday, today, last week, Mar 26, 3/26/04

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Powered by Vanilla. Made with Bootstrap.

(howto) Install and run Oncotator for the first time

Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin
edited November 30 in Oncotator documentation

1. Download the Oncotator package, the default datasources package, and (recommended) transcript override list from the Downloads page

Please note: Broadies who wish to run the installed Oncotator on the Broad cluster should follow the instructions here, instead of this page

Oncotator Download

Download the latest release here.

Default Datasource Corpus Download (January 26, 2015)

Download 14GB

Please note that this corpus should be used with Oncotator 1.4.x.x and above. Uniprot AA Pos annotations will not function properly with Oncotator 1.3.x.x and below.

Transcript override lists

We highly recommend that you download and use one of the below transcript override lists, especially if clinical applications of Oncotator. When running Oncotator, provide one of the below files with the -c parameter.

  • Download UniProt Exact Match For GENCODE v19, will give selection priority to transcripts with protein sequences that match the UniProt protein sequence exactly. This file can also be found in the Oncotator download at test/testdata/tx_exact_uniprot_matches.txt.

  • Download UniProt Exact Match + Clinical For GENCODE v19, this will give priority to known clinical protein changes. This file is a modification of the UniProt Exact Match (above). For more information about how this list was generated, please see the powerpoint presentation here

The Oncotator and default datasource corpus packages are simple tar files that can be expanded using the following commands:

$ tar zxvf oncotator-1.5.1.0.tar.gz
$ tar zxvf oncotator_v1_ds_Jan262015.tar.gz

This will produce two directories called oncotator-1.5.1.0 and oncotator_v1_ds_Dec112014, respectively. Move to the oncotator-1.5.1.0 directory by doing:

$ cd oncotator-1.5.1.0

2. Set up your Python environment and install dependencies

See the article on platform requirements for a full list of dependencies. This tutorial will show you how to use the virtual environment script we provide to set everything up automagically, and this tutorial will show you how to install dependencies manually if needed (or preferred).


3. Install Oncotator

Once you have installed all the necessary dependencies listed above, simply run the standard Python install script which is included with the Oncotator distribution.

$ python setup.py install

Two binaries (executable program files) named oncotator and initializeDatasource respectively will be installed into your Python's bin/ directory. You can test that they were installed by running e.g.:

$ oncotator -h 

to invoke the help / usage instructions. You can also do a test run of Oncotator on the Patient0.snp.maf.txt file provided with the Oncotator distribution (in the test/testdata/maflite/ directory) with the following command:

$ oncotator -v --db-dir /path/to/oncotator_v1_ds_Jan262015 test/testdata/maflite/Patient0.snp.maf.txt exampleOutput.tsv hg19

where you provide the location of the datasources using the --db-dir argument. You may need to adapt the file path for the Patient0.snp.maf.txt file depending on where you run this command from.

This will produce a new file named exampleOutput.tsv with the appropriate annotations, built against the hg19 reference.

Post edited by Geraldine_VdAuwera on

Geraldine Van der Auwera, PhD

Comments

  • pmintpmint Member Posts: 12

    An error occurred while installing Oncotator.

    error: Setup script exited with error: command 'gcc' failed with exit status 1

    What do i do ?

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    @pmint, have you looked at the compiler requirements in this document?

    Geraldine Van der Auwera, PhD

  • szhwatchszhwatch Member Posts: 1

    An error occurred while installing Oncotator.
    Yachtmaster

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    @szhwatch, we're going to need a little more information than that to help you. What did you try, what happened, ...?

    Geraldine Van der Auwera, PhD

  • xiuque88xiuque88 xiuque88Member Posts: 2

    Hi Geraldine,

    Thanks you and your team at Broad for providing a local installation of oncotator. Could you provide the checksum for the datasource corpus ? I want to ensure that the download went perfectly.

    Thanks!
    Cheers.

  • xiuque88xiuque88 xiuque88Member Posts: 2
    edited July 2014

    Good day,

    I was running the test set and a got a series of warnings :

                014-07-17`` 03:25:13,066 WARNING [oncotator.output.TcgaMafOutputRenderer:197] Entrez Gene ID was zero, but Hugo Symbol was not Unknown.  Is the HGNC and/or Transcript datasource complete?
    

    Could you please advice if this is expected?

    Cheers

  • joonlee3joonlee3 Member Posts: 1

    Hi Geraldine,

    I have the same issue. The link specified above is broken. Would you please provide a new one for me? I really appreciate it.

    Best,
    Joon

  • vguestvguest ChinaMember Posts: 1

    I was install oncotator and got a error: ERROR: Could not load pysam. Some features will be disabled (e.g. COSMIC annotations) and may cause Oncotator to fail. No handlers could be found for logger "root". I used the virtual environment script and the pysam-0.7.5 package have been successfully installed. I was confused why got this error when I test the oncotator. Can you give me some suggestion?

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    Hi @vguest‌,

    Can you post the output that was generated during the installation process?

    Geraldine Van der Auwera, PhD

  • chung2000chung2000 Member Posts: 8

    We love having the local installation of Oncotator because of the improved ease of use and convenience. We do trip up ourselves from time to time when we forgot whether the dozens of output files contain canonical or best effect annotations or that the --tx-mode flag was set to "EFFECT" or not. Can you add as a future enhancement a text string in the output file to label the output as either Canonical or Best Effect? The text string does not have to be in the column headers but wherever it is sensible for you to put it.

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    Hi @chung2000, we've put this in as a feature request. In the meantime, one way to keep track of this is to check the run logs.

    Geraldine Van der Auwera, PhD

  • chung2000chung2000 Member Posts: 8

    Thank you @Geraldine_VdAuwera. While the run logs are excellent records for when, where, and how the --tx-mode flag was set, all is lost when the Oncotator output files gets passed around to other people in our group or to other organizations because we rather not send the run logs to those people.

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    @chung2000 I understand, it is awkward to rely on log files. This will be implemented in an upcoming version -- we'll make an announcement when the next release is ready.

    Geraldine Van der Auwera, PhD

  • chung2000chung2000 Member Posts: 8
    edited October 2014

    If I want to obtain the latest version of the datasource corpus (more recent than the June 2014 release) where is it located? Thank you.

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    @chung2000 I don't believe there is a later version than June 2014 as of yet. When there is it will be made available on this page.

    Geraldine Van der Auwera, PhD

  • np3np3 Member Posts: 6

    Hi, when I run oncotator, I keep getting the error "pkg_resources.DistributionNotFound: natsort"
    I am new to command-line, but I have checked that the natsort package is present. Any help would be greaty appreciated. Thanks

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @np3 Try "pip install -U distribute" That has fixed this in the past.

  • np3np3 Member Posts: 6

    Thanks for the suggestion. Unfortunately, after doing this step and trying again, I received the same error message.

  • np3np3 Member Posts: 6

    We are using Ubuntu 12.04 LTS. Would that make a difference?

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @np3 Can you post the results of "pip freeze" ?

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @np3 I have been using Ubuntu 12.04 with no issues. Also, what version of python are you using?

  • np3np3 Member Posts: 6

    Python==2.7.3
    Brlapi==0.5.6
    Cython==0.21
    GnuPGInterface==0.3.2
    Jinja2==2.7.1
    Mako==0.5.0
    MarkupSafe==0.15
    MySQL-python==1.2.3
    PAM==0.4.2
    PIL==1.1.7
    Pillow==2.3.0
    Pyste==0.9.10
    Twisted-Core==11.1.0
    Twisted-Names==11.1.0
    Twisted-Web==11.1.0
    UniConvertor==1.1.4
    adium-theme-ubuntu==0.3.2
    apt-xapian-index==0.44
    apturl==0.5.1ubuntu3
    argparse==1.2.1
    biopython==1.62
    chardet==2.0.1
    chimerascan==0.4.5
    classicmenu-indicator==0.09
    command-not-found==0.2.44
    configglue==1.0
    configobj==4.7.2
    debtagshw==0.1
    decorator==3.3.2
    defer==1.0.2
    dirspec==3.0.0
    duplicity==0.6.18
    httplib2==0.7.2
    ipython==0.12.1
    jockey==0.9.7
    keyring==0.9.2
    language-selector==0.1
    launchpadlib==1.9.12
    lazr.restfulclient==0.12.0
    lazr.uri==1.0.3
    lockfile==0.8
    louis==2.3.0
    lxml==2.3.2
    matplotlib==1.1.1rc
    natsort==3.5.1
    nose==1.1.2
    numexpr==1.4.2
    numpy==1.6.1
    nvidia-common==0.0.0
    oauth==1.0.1
    onboard==0.97.1
    oneconf==0.2.8.1
    pandas==0.7.0
    pdfshuffler==0.6.0
    pexpect==2.3
    piston-mini-client==0.7.2
    protobuf==2.4.1
    pyOpenSSL==0.12
    pyPdf==1.13
    pycrypto==2.4.1
    pycups==1.9.61
    pycurl==7.19.0
    pyinotify==0.9.2
    pyparsing==1.5.2
    pysam==0.7.5
    pyserial==2.5
    pysmbc==1.0.13
    pysqlite==1.0.1
    python-apt==0.8.3ubuntu7.2
    python-dateutil==1.5
    python-debian==0.1.21ubuntu1
    python-virtkey==0.60.0
    pytz==2011k
    pyxdg==0.19
    pyzmq==2.1.11
    reportlab==2.5
    rhythmbox-ubuntuone==4.2.0
    rpy==1.0.3
    scikits.statsmodels==0.3.1
    scipy==0.9.0
    sessioninstaller==0.0.0
    simplegeneric==0.7
    simplejson==2.3.2
    software-center-aptd-plugins==0.0.0
    sympy==0.7.1.rc1
    system-service==0.1.6
    tables==2.3.1
    tornado==2.1
    ubuntuone-couch==0.3.0
    ubuntuone-installer==3.0.2
    ubuntuone-storage-protocol==3.0.2
    ufw==0.31.1-1
    unattended-upgrades==0.1
    unity-lens-video==0.3.5
    unity-scope-video-remote==0.3.5
    urlgrabber==3.9.1
    usb-creator==0.2.23
    virtualenv==1.11.6
    wadllib==1.3.0
    wsgiref==0.1.2
    xdiagnose==2.5.3
    xkit==0.0.0
    xlrd==0.9.2
    yum-metadata-parser==1.1.2
    zope.interface==3.6.1

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @chung2000‌ @Geraldine_VdAuwera‌ I have posted the Sept 17 corpus here.

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @np3 Though I have newer packages that is very similar to my configuration. Usually the issue here is an incompatible version of distribute or setuptools with the version of python. Are you using a virtual environment? In other words, did you run scripts/create_oncotator_venv.sh? Apologies for basic questions. Also, use @LeeTL1220 to get a faster response.

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @np3 For a different project, I just got the same error as you and I just ran pip uninstall distribute and it worked.

  • ramiro2kramiro2k Member Posts: 8

    Hello, I have problems making oncotator work, the first one is pysam 0.7.5, I can install 0.7.8 but for 0.7.5 I get:

    NameError: name 'sys_platform' is not defined

    /tmp/pip_build_rbarrant/pysam/distribute-0.6.34-py2.7.egg

    Traceback (most recent call last):

    File "", line 17, in

    File "/tmp/pip_build_rbarrant/pysam/setup.py", line 131, in

    use_setuptools()
    

    File "distribute_setup.py", line 152, in use_setuptools

    return _do_download(version, download_base, to_dir, download_delay)
    

    File "distribute_setup.py", line 132, in _do_download

    _build_egg(egg, tarball, to_dir)
    

    File "distribute_setup.py", line 123, in _build_egg

    raise IOError('Could not build the egg.')
    

    IOError: Could not build the egg.


    Cleaning up...
    Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_rbarrant/pysam
    Storing debug log for failure in /users/r/b/rbarrant/.pip/pip.log
    bash-3.2$

    Can things work with 0.7.8 or do I REALLY need 0.7.5?

    Thanks in advance,
    Ramiro

  • np3np3 Member Posts: 6

    @LeeTL1220‌ Thanks for the info. I previously used "pip install virtualenv". I now tried your suggestion for installing the VE. I also uninstalled distribute before and after doing so, and also reinstalled it at the end. I could not get oncotator to work at any of these steps. I should say I am very new to Linux, so any basic advice is appreciated.

    Btw, I used the command "./oncotator -h" from the "bin" directory. Is this correct?

    The next step would be to try to use the new Corpus. To do so, do I need to uninstall/remove the old one?

    Thanks for your help.

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    Geraldine Van der Auwera, PhD

  • np3np3 Member Posts: 6

    @Geraldine_VdAuwera‌ Sorry for the late reply. Yes, that is guide I used. Thanks.

  • chung2000chung2000 Member Posts: 8

    @np3, if your installation is like mine, you would use the the command "./oncotator -h" from the "oncotator-1.3.0.0" directory.

    Also, you don't need to remove the old Corpus unless you want to recover the disk space that it is occupying.

  • JeremySJeremyS ParisMember Posts: 1

    Hello,
    I have an error while using VCF file : ValueError: could not convert string to float:
    I have install the patched version of PyVCF.

    Regards

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @JeremyS‌ Did you use the create_oncotator_venv.sh script to create a virtual environment and run oncotator from there?

  • inkink Member Posts: 7

    Having spent a few days making Oncotator to work, I want to share my solution. I think the main problem stems from the fact that Oncotator uses distribute-0.6.15 which does not understand wheel format (new Python packaging). Therefore if you happen to have some of the required packages installed as dist-info rather than egg-info you are likely to run into trouble (check your lib/python2.7/site-packages/ )

    We are running SL 6.4 on our cluster so no Python 2.7 in the distro and we are using modules.
    1/ installed python/2.7.9; created a module for python/2.7.9
    2/ installed numpy and scipy with Intel MKL support; after that

    ~> module load intel-mkl/11.0u1 python/2.7.9
    ~> pip freeze
    Cython==0.21.2
    h5py==2.4.0
    nose==1.3.4
    numpy==1.9.1
    scipy==0.15.1
    virtualenv==12.0.5
    wsgiref==0.1.2
    

    This is a minimal install. Uninstall nose before proceeding further (nose was needed for testing numpy)

    pip uninstall nose
    

    After that you could proceed with virtualenv setup as recommended elsewhere. But we prefer modules.

    3/ created oncotator module containing this

    set base /apps/oncotator/1.4.1.0
    prepend-path PYTHONPATH $base/lib/python2.7/site-packages
    prepend-path PATH $base/bin
    

    4/ downloaded and unpacked oncotator 1.4.1.0

    5/ adapted oncotator-1.4.1.0/scripts/create_oncotator_venv.sh to make my own script for installing prerequisites

    #!/bin/bash
    module load intel-mkl/11.0u1 python/2.7.9 oncotator/1.4.1.0
    TARGET=/apps/oncotator/1.4.1.0
    #C_PACKAGE_LIST="biopython cython numpy pandas sqlalchemy"
    C_PACKAGE_LIST="biopython pandas sqlalchemy"
    for C_PACKAGE in $C_PACKAGE_LIST; do
        pip install --install-option="--prefix=$TARGET" --no-use-wheel $C_PACKAGE
    done
    PACKAGE_LIST="bcbio-gff nose shove python-memcached natsort more-itertools enum34"
    for PACKAGE in $PACKAGE_LIST; do
        pip install --install-option="--prefix=$TARGET" -U --no-use-wheel $PACKAGE
    done
    wget https://github.com/elephanthunter/PyVCF/archive/master.zip
    mv master master.zip
    unzip master.zip 
    cd PyVCF-master 
    python2.7 setup.py install --prefix=$TARGET
    cd ..
    

    cython and numpy are removed from C_PACKAGE_LIST because we have them already. Should not be any harm if there are not removed.

    6/ installed oncotator itself

    python2.7 setup.py install --prefix=/apps/oncotator/1.4.1.0
    

    7/ after that loading

    module load intel-mkl/11.0u1 python/2.7.9 oncotator/1.4.1.0
    

    should make oncotator available on the command line.

    8/ it should be also fine to do the following

    module unload oncotator/1.4.1.0
    pip install nose
    

    i.e. install nose back into the new python because nose is such an essential package. Oncotator should still be happy with its nose egg install.

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    @ink, thanks for reporting your solution.

    Geraldine Van der Auwera, PhD

  • woodydonwoodydon TaipeiMember Posts: 17

    I unzipped oncotator_v1_ds_Jan262015.tar.gz and found that the folder name was oncotator_v1_ds_Jan262014 instead of oncotator_v1_ds_Jan262015. Was it simply a typo?

  • chung2000chung2000 Member Posts: 8

    @woodydon said:

    I unzipped oncotator_v1_ds_Jan262015.tar.gz and found that the folder name was oncotator_v1_ds_Jan262014 instead of oncotator_v1_ds_Jan262015. Was it simply a typo?

    I found the same thing but was able to run Oncotator with the new corpus to get the upgraded 1000 genome and dbDNP data. Most likely the folder name had a typo, as is usually the case at the beginning of any year.

  • Geraldine_VdAuweraGeraldine_VdAuwera Administrator, Dev Posts: 10,677 admin

    Yes that looks like a typo, sorry about that.

    Geraldine Van der Auwera, PhD

  • asoskinsasoskins Imperial College, Lonfon, UKMember Posts: 7

    Hello, I've installed oncotator/1.5.1.0 and try to run it with python/2.7.3 on my vcf files with the following command line:
    oncotator -i VCF --db-dir $TMPDIR/oncotator_db -o VCF $TMPDIR/merged.vcf $TMPDIR/annotated.vcf hg19

    I keep getting the following error message:

    2015-04-28 20:17:21,924 ERROR [oncotator.output.VcfOutputRenderer:222] Traceback (most recent call last):
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/VcfOutputRenderer.py", line 214, in _renderSortedTsv
    recordBuilder = self._parseRecordBuilder(m, recordBuilder, dataManager, inferGenotypes)
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/VcfOutputRenderer.py", line 276, in _parseRecordBuilder
    recordBuilder.addInfo(sampleName, ID, num, dataType, val, isSplit)
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/RecordBuilder.py", line 291, in addInfo
    val = self._map(MutUtils.str2bool, self._fixVal(val, isSplit)) # convert the value to boolean
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/RecordBuilder.py", line 97, in _map
    return [func(v) if v not in bad else None for v in iterable]
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/utils/MutUtils.py", line 222, in str2bool
    return v.lower() in ("yes", "true", "t", "1", "y")
    AttributeError: 'NoneType' object has no attribute 'lower'

    2015-04-28 20:17:21,924 ERROR [oncotator.output.VcfOutputRenderer:223] Error at mutation 1 ['1', '19865', '19865']:

    Can you please help with this issue?
    Thanks in advance,
    Alona

  • sherey65sherey65 SeattleMember Posts: 3

    I recently downloaded Oncotator in order to use it on open-access TCGA MAF files. I kept only the first 34 columns of the MAF file, in order to remove any additional columns that may have been added by one of the TCGA data centers, and tried to input it to Oncotator using the --input_format=MAFLITE option. It seems like the two "dbSNP" columns in the TCGA MAF format caused a problem, and I got this error:

    Traceback (most recent call last):
    File "/tools/bin/Oncotator", line 9, in
    load_entry_point('Oncotator==v1.5.1.0', 'console_scripts', 'Oncotator')()
    File "/tools/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Oncotator.py", line 292, in main
    annotator.annotate()
    File "/tools/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 403, in annotate
    filename = self._outputRenderer.renderMutations(mutations, metadata=metadata, comments=comments)
    File "/tools/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/output/TcgaMafOutputRenderer.py", line 283, in renderMutations
    raise e
    oncotator.DuplicateAnnotationException.DuplicateAnnotationException: 'Attempting to create an annotation multiple times (dbSNP_RS) with old, new values of (, rs377395363)'

    Removing the dbSNP_RS and dbSNP_Val_Status columns eliminated this problem.

    Another issue I noticed is that column 12 in the annotated output ("Tumor_Seq_Allele1") is actually a copy of the preceding column ("Reference_Allele"), while column 13 ("Tumor_Seq_Allele2") is almost always the same as the input column 13, but sometimes different. Also the NCBI_Build in column #4 gets reset to "UNKNOWN" (although column #196 has the correct build information).

    I was expecting that columns 1-34 in the original MAF would be the same in the output annotated MAF -- is there a reason why this is not the case?

    thanks,

    Sheila

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    If you upgrade to Oncotator 1.5.3.0, I believe this issue has been fixed.

    @asoskins said:
    Hello, I've installed oncotator/1.5.1.0 and try to run it with python/2.7.3 on my vcf files with the following command line:
    oncotator -i VCF --db-dir $TMPDIR/oncotator_db -o VCF $TMPDIR/merged.vcf $TMPDIR/annotated.vcf hg19

    I keep getting the following error message:

    2015-04-28 20:17:21,924 ERROR [oncotator.output.VcfOutputRenderer:222] Traceback (most recent call last):
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/VcfOutputRenderer.py", line 214, in _renderSortedTsv
    recordBuilder = self._parseRecordBuilder(m, recordBuilder, dataManager, inferGenotypes)
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/VcfOutputRenderer.py", line 276, in _parseRecordBuilder
    recordBuilder.addInfo(sampleName, ID, num, dataType, val, isSplit)
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/RecordBuilder.py", line 291, in addInfo
    val = self._map(MutUtils.str2bool, self._fixVal(val, isSplit)) # convert the value to boolean
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/RecordBuilder.py", line 97, in _map
    return [func(v) if v not in bad else None for v in iterable]
    File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/utils/MutUtils.py", line 222, in str2bool
    return v.lower() in ("yes", "true", "t", "1", "y")
    AttributeError: 'NoneType' object has no attribute 'lower'

    2015-04-28 20:17:21,924 ERROR [oncotator.output.VcfOutputRenderer:223] Error at mutation 1 ['1', '19865', '19865']:

    Can you please help with this issue?
    Thanks in advance,
    Alona

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @sherey65 Yes, removing the dbSNP annotations in the input file should eliminate the issue. As per the TCGA MAF spec, Reference_Allele and Tumor_Seq_Allele1 should be the same. If Tumor_Seq_Allele1 is the same as Tumor_Seq_Allele2, then you should check your input file. There are specific instructions for your use case (TCGA MAF in and TCGA MAF out) on the following page: http://gatkforums.broadinstitute.org/discussion/4177/what-input-files-can-i-annotate-with-oncotator#latest

  • mwongmwong SingaporeMember Posts: 1

    Running the test file throws this error:

    2015-05-22 05:52:49,823 INFO [oncotator.Oncotator:235] Oncotator v1.5.1.0
    2015-05-22 05:52:49,823 INFO [oncotator.Oncotator:236] Args: Namespace(cache_url=None, canonical_tx_file=None, dbDir='oncotator_db', default_cli=[], default_config=None, genome_build='hg19', infer_genotypes='false', infer_onps=False, input_file='Patient0.snp.maf.txt', input_format='MAFLITE', log_name='oncotator.log', noMulticore=False, output_file='myOutputFile.maf.annotated', output_format='TCGAMAF', override_cli=[], override_config=None, prepend=False, read_only_cache=False, skip_no_alt=False, tx_mode='CANONICAL', verbose=6)
    2015-05-22 05:52:49,823 INFO [oncotator.Oncotator:237] Log file: /mergedcancerdata/oncotator.log
    2015-05-22 05:52:49,823 WARNING [oncotator.Oncotator:243] ngslib module not installed. Will be unable to annotate with BigWig datasources.
    2015-05-22 05:52:49,824 INFO [oncotator.input.MafliteInputMutationCreator:115] Initializing a maflite file with the following header: ['build', 'chr', 'start', 'end', 'ref_allele', 'tum_allele1', 'tum_allele2', 'tumor_barcode', 'normal_barcode', 't_lod_fstar', 't_alt_count', 't_ref_count']
    2015-05-22 05:52:49,825 INFO [oncotator.output.TcgaMafOutputRenderer:90] Building alternative keys dictionary...
    2015-05-22 05:52:49,825 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for oncotator_db/1000genome_db/hg19/1000genome_db.config
    2015-05-22 05:52:49,825 INFO [oncotator.DatasourceFactory:304] Creating pool
    2015-05-22 05:52:49,835 INFO [oncotator.DatasourceFactory:307] Pool created

    Traceback (most recent call last):
    File "/usr/local/bin/oncotator", line 9, in
    load_entry_point('Oncotator==v1.5.1.0', 'console_scripts', 'oncotator')()
    File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Oncotator.py", line 288, in main
    other_opts=determineOtherOptions(args), annotating_type=annotating_type)
    File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/utils/RunSpecificationFactory.py", line 118, in create_run_spec
    datasource_list = DatasourceFactory.createDatasources(datasourceDir, genomeBuild, isMulticore=isMulticore, numCores=numCores, tx_mode=tx_mode)
    File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 294, in createDatasources
    result = DatasourceFactory._createDatasourcesMulticore(numCores, dsQueueList)
    File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 317, in _createDatasourcesMulticore
    result.append(DatasourceFactory.createDatasourceGivenTuple(dsTuple))
    File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 106, in createDatasourceGivenTuple
    return DatasourceFactory.createDatasource(configTuple[0], configTuple[1])
    File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 100, in createDatasource
    return DatasourceFactory.createDatasourceFromConfigParser(configParser, leafDir)
    File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 186, in createDatasourceFromConfigParser
    match_mode=configParser.get('general', 'match_mode'))
    File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/datasources/TabixIndexedVcfDatasource.py", line 84, in init
    self.vcf_reader = vcf.Reader(filename=src_file, strict_whitespace=True)
    File "/usr/local/lib/python2.7/dist-packages/PyVCF-0.6.7-py2.7-linux-x86_64.egg/vcf/parser.py", line 288, in init
    self._parse_metainfo()
    File "/usr/local/lib/python2.7/dist-packages/PyVCF-0.6.7-py2.7-linux-x86_64.egg/vcf/parser.py", line 304, in _parse_metainfo
    line = self.reader.next()
    File "/usr/local/lib/python2.7/dist-packages/PyVCF-0.6.7-py2.7-linux-x86_64.egg/vcf/parser.py", line 268, in
    self.reader = (line.strip() for line in self._reader if line.strip())
    File "/usr/lib/python2.7/gzip.py", line 451, in readline
    c = self.read(readsize)
    File "/usr/lib/python2.7/gzip.py", line 258, in read
    if not self._read(readsize):
    File "/usr/lib/python2.7/gzip.py", line 293, in _read
    self._read_gzip_header()
    File "/usr/lib/python2.7/gzip.py", line 205, in _read_gzip_header
    self._read_exact(struct.unpack("<H", self._read_exact(2)))
    File "/usr/lib/python2.7/gzip.py", line 185, in _read_exact
    data = self.fileobj.read(n)
    TypeError: an integer is required

  • XBonXBon SwitzerlandMember Posts: 17

    Hello!
    I recently downloaded oncotator v1.5.3.0 and the newest database bundle (Jan262014) and I cannot annotate my samples. Oncotator works well and parses through my input file, but there is no annotations at the end. Apparently the datasources are not recognized:

    2015-05-26 12:34:23,695 INFO [oncotator.Oncotator:235] Oncotator v1.5.3.0
    2015-05-26 12:34:23,696 INFO [oncotator.Oncotator:236] Args: Namespace(cache_url=None, canonical_tx_file=None, dbDir='/data2/unige/dmg/xbonilla/oncotator_v1_ds_Jan262015/', default_cli=[], default_config=None, genome_build='hg19', infer_genotypes='false', infer_onps=True, input_file='call_stats_5-VS018-T1.out', input_format='MAFLITE', log_name='5-VS018-T1.ann.log', noMulticore=False, output_file='5-VS018-T1.maf.annotated', output_format='TCGAMAF', override_cli=[], override_config=None, prepend=False, read_only_cache=False, skip_no_alt=False, tx_mode='CANONICAL', verbose=6)
    2015-05-26 12:34:23,696 WARNING [oncotator.Oncotator:243] ngslib module not installed. Will be unable to annotate with BigWig datasources.
    2015-05-26 12:34:23,696 INFO [oncotator.utils.RunSpecificationFactory:108] Output file order may be different than input file because we're combining SNPs into ONPs
    2015-05-26 12:34:34,261 INFO [oncotator.input.MafliteInputMutationCreator:115] Initializing a maflite file with the following header: ['contig', 'position', 'context', 'ref_allele', 'alt_allele', 'tumor_name', 'normal_name', 'score', 'dbsnp_site', 'covered', 'power', 'tumor_power', 'normal_power', 'total_pairs', 'improper_pairs', 'map_Q0_reads', 't_lod_fstar', 'tumor_f', 'contaminant_fraction', 'contaminant_lod', 't_ref_count', 't_alt_count', 't_ref_sum', 't_alt_sum', 't_ref_max_mapq', 't_alt_max_mapq', 't_ins_count', 't_del_count', 'normal_best_gt', 'init_n_lod', 'n_ref_count', 'n_alt_count', 'n_ref_sum', 'n_alt_sum', 'judgement']
    2015-05-26 12:34:34,565 INFO [oncotator.input.OnpCombiner:77] Merging adjacent snps from the same samples into ONPs
    2015-05-26 12:34:34,582 INFO [oncotator.output.TcgaMafOutputRenderer:90] Building alternative keys dictionary...
    2015-05-26 12:34:34,584 WARNING [oncotator.DatasourceFactory:260] %s does not exist, so there will be no datasources.
    2015-05-26 12:34:34,584 INFO [oncotator.DatasourceFactory:325] No datasources to initialize
    2015-05-26 12:34:34,584 INFO [oncotator.cache.DummyCache:57] No cache specified. All cache attempts will be listed as cache misses.
    2015-05-26 12:34:34,585 INFO [oncotator.Annotator:392] Annotating with 0 datasources: Oncotator v1.5.3.0 |
    2015-05-26 12:34:34,585 INFO [oncotator.output.TcgaMafOutputRenderer:218] TCGA MAF output file: newmutect_newdatabase_5-VS018-T1.maf.annotated
    2015-05-26 12:34:34,586 INFO [oncotator.output.TcgaMafOutputRenderer:219] Render starting...
    2015-05-26 12:34:34,586 WARNING [oncotator.Annotator:466] THERE ARE NO DATASOURCES REGISTERED
    2015-05-26 12:34:34,588 INFO [oncotator.utils.SampleNameSelector:90] Sample name is the concatenation of the normal_name and tumor_name columns.
    2015-05-26 12:34:34,955 WARNING [oncotator.input.OnpQueue:196] Mutations are not sorted by start position, this may cause unexpected behavior or increased memory requirements. It is recommended that your sort any files that youusing with --infer-onps by position and sample name.
    2015-05-26 12:34:35,685 INFO [oncotator.output.TcgaMafOutputRenderer:276] Rendered 1000 mutations.
    ...

    I was wondering if I have to initialize one by one the databases from the bundle, or if there is a way of getting oncotator to recognize them since they are already in the right format? I am not sure what I missed in the installation process.
    Thank you in advance and I'm sorry for the newbie question!

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @mwong I think v1.5.3.0 fixes this issue.

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @XBon I would need to see your directory structure, but I think you need to add one more dir to the end of your db-dir. db-dir should specify the parent directory of ref_hg/, hgnc/, dbnsfp/, etc.

  • ravichavravichav United StatesMember Posts: 20

    I ran oncotator1.5.1.0 on a HC called VCF using the command

    oncotator -i VCF -o VCF --db-dir /path_to/ngstools/oncotator_v1_ds_Jan262014/ Project_bwa_GatkHC.vcf Project_bwa_GatkHC_oncotator.vcf hg19

    The script ran for about 214000 mutations and threw an exception. Pasting the error below. Can someone help me understand this error?

    2015-06-11 13:11:58,655 INFO [oncotator.output.OutputDataManager:194] Wrote 214000 mutations to tsv.

    Traceback (most recent call last):
    File "/home/Pythonmodules/bin/oncotator", line 9, in
    load_entry_point('Oncotator==v1.5.1.0', 'console_scripts', 'oncotator')()
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Oncotator.py", line 292, in main
    annotator.annotate()
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 403, in annotate
    filename = self.outputRenderer.renderMutations(mutations, metadata=metadata, comments=comments)
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/output/VcfOutputRenderer.py", line 118, in renderMutations
    dataManager = OutputDataManager(self.configTable, mutations, comments, metadata, path)
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/output/OutputDataManager.py", line 92, in __init_
    self.chroms, self.sampleNames, self.filename = self._writeMuts2Tsv(self.mutations, path)
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/output/OutputDataManager.py", line 160, in _writeMuts2Tsv
    for mut in muts:
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 414, in _applyManualAnnotations
    for m in mutations:
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 422, in _applyDefaultAnnotations
    for m in mutations:
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 470, in _annotate_mutations_using_datasources
    for m in mutations:
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/input/VcfInputMutationCreator.py", line 306, in createMutations
    sampleMut = self._addGenotypeData2Mutation(sampleMut, record, index)
    File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/input/VcfInputMutationCreator.py", line 130, in _addGenotypeData2Mutation
    val = genotypeData[ID][index]
    IndexError: list index out of range

  • alaganaalagana Member Posts: 4

    Hello,
    I am having the same issue as @mwong. I tried both versions 1.5.1.0 and 1.5.3.0 and it stops on the error "TypeError: an integer is required" as soon as it starts. Any ideas how to fix this?

    Alessandro

    =====

    2015-06-17 16:15:01,636 INFO [oncotator.Oncotator:237] Log file: /data1/alessandro/projects/tools/oncotator-1.5.3.0/oncotator.log
    2015-06-17 16:15:01,636 INFO [oncotator.utils.RunSpecificationFactory:108] no prepend specified when output is not TCGAMAF. Ignoring and proceeding.
    2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/ccle_by_gp/hg19/ccle_by_gp.config
    2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/tcgascape/hg19/tcgascape.config
    2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/esp6500SI_v2_coverage/hg19/esp6500SI_v2_coverage.config
    2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/esp6500SI_v2/hg19/esp6500SI_v2.config
    2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/achilles/hg19/achilles.config
    2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/oreganno/hg19/oreganno.config
    2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/hgnc/hg19/hgnc.config
    2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/ccle_by_gene/hg19/ccle_by_gene.config
    2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/clinvar/hg19/clinvar.config
    2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/dna_repair_genes/hg19/dna_repair_genes.config
    2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/cancer_gene_census/hg19/cancer_gene_census.config
    2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/dbsnp/hg19/dbsnp.config
    2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/cosmic_tissue/hg19/cosmic_tissue.config
    2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/gencode_out2/hg19/gencode_out2.config
    2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/uniprot_aa_annotation/hg19/uniprot_aa_annotation.config
    2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/ref_hg/hg19/ref_hg.config
    2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/cosmic_fusion/hg19/cosmic_fusion.config
    2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/uniprot_aa_xform/hg19/uniprot_aa_xform.config
    2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/tumorscape/hg19/tumorscape.config
    2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/mutsig/hg19/mutsig.config
    2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/cosmic/hg19/cosmic.config
    2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/1000genome_db/hg19/1000genome_db.config
    2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/dbNSFP_ds/hg19/dbNSFP_ds.config
    2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/familial/hg19/familial.config
    2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/gencode_xrefseq/hg19/gencode_xrefseq.config
    2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/simple_uniprot/hg19/simple_uniprot.config
    2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/so_terms/hg19/so_terms.config
    2015-06-17 16:15:01,650 INFO [oncotator.DatasourceFactory:304] Creating pool
    2015-06-17 16:15:01,658 INFO [oncotator.DatasourceFactory:307] Pool created
    Traceback (most recent call last):
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/bin/oncotator", line 9, in
    load_entry_point('Oncotator==1.5.3.0', 'console_scripts', 'oncotator')()
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/Oncotator.py", line 288, in main
    other_opts=determineOtherOptions(args), annotating_type=annotating_type)
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/utils/RunSpecificationFactory.py", line 118, in create_run_spec
    datasource_list = DatasourceFactory.createDatasources(datasourceDir, genomeBuild, isMulticore=isMulticore, numCores=numCores, tx_mode=tx_mode)
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 294, in createDatasources
    result = DatasourceFactory._createDatasourcesMulticore(numCores, dsQueueList)
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 317, in _createDatasourcesMulticore
    result.append(DatasourceFactory.createDatasourceGivenTuple(dsTuple))
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 106, in createDatasourceGivenTuple
    return DatasourceFactory.createDatasource(configTuple[0], configTuple[1])
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 100, in createDatasource
    return DatasourceFactory.createDatasourceFromConfigParser(configParser, leafDir)
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 186, in createDatasourceFromConfigParser
    match_mode=configParser.get('general', 'match_mode'))
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/datasources/TabixIndexedVcfDatasource.py", line 84, in init
    self.vcf_reader = vcf.Reader(filename=src_file, strict_whitespace=True)
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/PyVCF-0.6.4-py2.7-linux-x86_64.egg/vcf/parser.py", line 283, in init
    self._parse_metainfo()
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/PyVCF-0.6.4-py2.7-linux-x86_64.egg/vcf/parser.py", line 299, in _parse_metainfo
    line = self.reader.next()
    File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/PyVCF-0.6.4-py2.7-linux-x86_64.egg/vcf/parser.py", line 263, in
    self.reader = (line.strip() for line in self._reader if line.strip())
    File "/usr/lib/python2.7/gzip.py", line 451, in readline
    c = self.read(readsize)
    File "/usr/lib/python2.7/gzip.py", line 258, in read
    if not self._read(readsize):
    File "/usr/lib/python2.7/gzip.py", line 293, in _read
    self._read_gzip_header()
    File "/usr/lib/python2.7/gzip.py", line 205, in _read_gzip_header
    self._read_exact(struct.unpack("<H", self._read_exact(2)))
    File "/usr/lib/python2.7/gzip.py", line 185, in _read_exact
    data = self.fileobj.read(n)
    TypeError: an integer is required

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @alagana @mwong Did you guys use the oncotator create_oncotator_venv.sh script?

  • varshavarsha FloridaMember Posts: 37

    Hi guys, I downloaded the virtual environment successfully and was able to run the create_oncotator_venv.sh script but I am not able to run - python setup.py install. I tried this inside the virtual environment as well but it gave me the error -

    ** File "numpy/core/setup.py", line 42, in check_types
    7.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
    File "numpy/core/setup.py", line 293, in check_types
    SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.**

    I also tried - apt-get install python-dev. I verified if I have Numpy installed in Python and I do have it. Please let me know how to fix this and run oncotator. I am not sure what I am missing here. Thank you!

  • daihdaih indianaMember Posts: 8
    edited September 2015

    Hi,
    When running oncotator, I got the following error message like this: ImportError: No module named _sqlite3
    I'm not good at python. Can your experts solve this problem?
    And here's my command line:
    oncotator -v --db-dir $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014 --input_format=VCF $HOME/K104416/GATK/K104416.output.filter.vcf Output.tsv hg19

    And I'm sure sqlite version 3.08.11.1 had been loaded.

  • daihdaih indianaMember Posts: 8

    @daih said:
    Hi,
    When running oncotator, I got the following error message like this: ImportError: No module named _sqlite3
    I'm not good at python. Can your experts solve this problem?
    And here's my command line:
    oncotator -v --db-dir $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014 --input_format=VCF $HOME/K104416/GATK/K104416.output.filter.vcf Output.tsv hg19

    And I'm sure sqlite version 3.08.11.1 had been loaded.

    Never mind the above problem. Now it worked after I re-started my putty.
    But here's another question. I don't know why no datasources to initialize. It still can render all mutations.

    2015-09-13 17:52:47,220 WARNING [oncotator.Oncotator:243] ngslib module not installed. Will be unable to annotate with BigWig datasources.
    2015-09-13 17:52:47,387 INFO [oncotator.output.TcgaMafOutputRenderer:90] Building alternative keys dictionary...
    2015-09-13 17:52:47,389 WARNING [oncotator.DatasourceFactory:260] %s does not exist, so there will be no datasources.
    2015-09-13 17:52:47,389 INFO [oncotator.DatasourceFactory:325] No datasources to initialize
    2015-09-13 17:52:47,389 INFO [oncotator.cache.DummyCache:57] No cache specified. All cache attempts will be listed as cache misses.
    2015-09-13 17:52:47,389 INFO [oncotator.Annotator:392] Annotating with 0 datasources: Oncotator v1.5.1.0 |
    2015-09-13 17:52:47,433 INFO [oncotator.output.TcgaMafOutputRenderer:218] TCGA MAF output file: Output.tsv
    2015-09-13 17:52:47,433 INFO [oncotator.output.TcgaMafOutputRenderer:219] Render starting...
    2015-09-13 17:52:47,433 WARNING [oncotator.Annotator:466] THERE ARE NO DATASOURCES REGISTERED
    2015-09-13 17:52:48,702 INFO [oncotator.output.TcgaMafOutputRenderer:276] Rendered 1000 mutations.
    ...

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @varsha Did you install python-dev after your created the virtualenv?

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @daih You should see the datasource directories in $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014. For example, the directory $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014/hgnc/hg19 should exist. Perhaps, you have an extra subdirectory?

  • daihdaih indianaMember Posts: 8

    @LeeTL1220 said:
    daih You should see the datasource directories in $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014. For example, the directory $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014/hgnc/hg19 should exist. Perhaps, you have an extra subdirectory?

    these are what I alrealdy have, is there anything else?

    $ ls hgnc/hg19
    hgnc.config hgnc_download_Sept172014.bin hgnc_download_Sept172014.tsv

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @daih Can you provide ls -la $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014? It is claiming that this directory does not exist.

  • daihdaih indianaMember Posts: 8

    @LeeTL1220 said:
    daih Can you provide ls -la $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014? It is claiming that this directory does not exist.

    total 808
    drwxr-xr-x 29 daih faculty 4096 Sep 10 15:13 .
    drwxr-xr-x 3 daih faculty 4096 Sep 10 14:24 ..
    drwxr-xr-x 3 daih faculty 4096 Jan 23 2015 1000genome_db
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 achilles
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 cancer_gene_census
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 ccle_by_gene
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 ccle_by_gp
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 clinvar
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 cosmic
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 cosmic_fusion
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 cosmic_tissue
    drwxr-xr-x 3 daih faculty 4096 Jun 9 2014 dbNSFP_ds
    drwxr-xr-x 3 daih faculty 4096 Jan 23 2015 dbsnp
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 dna_repair_genes
    drwxr-xr-x 3 daih faculty 4096 Feb 7 2014 esp6500SI_v2
    drwxr-xr-x 3 daih faculty 4096 Feb 7 2014 esp6500SI_v2_coverage
    -rw-r--r-- 1 daih faculty 701609 Sep 10 15:13 exampleOutput.tsv
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 familial
    drwxr-xr-x 3 daih faculty 4096 Sep 10 15:36 gencode_out2
    drwxr-xr-x 4 daih faculty 4096 Nov 25 2014 gencode_xrefseq
    drwxr-xr-x 3 daih faculty 4096 Sep 17 2014 hgnc
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 mutsig
    -rw-r--r-- 1 daih faculty 2305 Sep 10 15:13 oncotator.log
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 oreganno
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 ref_hg
    drwxr-xr-x 3 daih faculty 4096 Dec 5 2014 simple_uniprot
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 so_terms
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 tcgascape
    drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 tumorscape
    drwxr-xr-x 3 daih faculty 4096 Dec 5 2014 uniprot_aa_annotation
    drwxr-xr-x 4 daih faculty 4096 Dec 15 2014 uniprot_aa_xform

  • varshavarsha FloridaMember Posts: 37

    Hi could you please send me the link to download the oncotator-1.5.3.0 version? Thank you.

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @varsha Just updated to the latest.

  • rkanwarrkanwar Member Posts: 2
    edited September 2015

    Hi,
    Is it possible to host oncotator with a web front end [like the one on Broad's website] locally ? I want to use the JSON API to annotate variants from other hosts. Thanks.

  • ravichavravichav United StatesMember Posts: 20

    IS there a way I can manually pick the annotations I want and not all the tracks by default?

  • mikedmiked Member Posts: 18

    Hello,

    I'm able to run Oncotator v1.5.1.0 successfully on many samples. I'm using database source oncotator_v1_ds_Jan262015.tar.gz
    The input file is a VCF and I'm using SIMPLE_TSV format as output.

    I'm not able to find the COSMIC ID for the variants ( if exists in COSMIC ). Am I missing something obvious?

    Please advise. Thanks.

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @miked Unfortunately, COSMIC IDs are not provided.

  • inkink Member Posts: 7

    Unfortunately there are still issues installing Oncotator. The principal issue related to v 1.8.0.0 is that it requires pysam 0.7.5 which is not available from pypi. As a result oncotator setup gives an error about missing pysam. The dependencies script installs everything but pysam in fact needlessly upgrading to the latest pysam version even if you happened to have the right one in the first place.
    Here is my recipe:
    1/ download and install pysam 0.7.5
    https://pysam.googlecode.com/files/pysam-0.7.5.tar.gz
    2/ install dependencies as listed in scripts/create_oncotator_venv.sh BUT change ngslib to "ngslib==1.1.9" where necessary because installing newer version will upgrade your pysam which you want to avoid
    3/ install oncotator itself (python setup.py install)
    Hope this helps

  • arsalaarsala PakistanMember Posts: 1

    Hello

    I have downloaded and installed oncotator as well as all the dependencies. The installation of oncotator has been confirmed by running the command "oncotator -h" to invoke usage instructions.

    Now I am doing test run of oncotator with the following command: $ oncotator -v --db-dir /path/to/oncotator_v1_ds_Jan262015 test/testdata/maflite/Patient0.snp.maf.txt exampleOutput.tsv hg19

    and I am getting this error: TypeError: _open() got an unexpected keyword argument 'encoding'

    Please help me out.

  • ANswer_777ANswer_777 Member Posts: 5

    @ramiro2k ,@ink
    I confronted with same problem. I just modify the setup.py with editing 'pysam==0.7.5' to 'pysam' , pyvcf >= 0.6.8' to 'pyvcf == 0.6.8'
    ps:The version in Github might fix some underlying bugs (https://github.com/broadinstitute/oncotator).

  • dtcdtcdtcdtcdtcdtcdtcdtc Member Posts: 3

    @arsala I have the exactly same problem, some one can help me out ?

  • LeeTL1220LeeTL1220 Arlington, MAMember, Broadie, Dev Posts: 73 ✭✭

    @arsala @dtcdtcdtcdtc Can you guys run with the latest version (1.9.1.0) and the latest default datasources? Also, are you running python 2.7?

  • xiangchunxiangchun ChinaMember Posts: 1

    When I used oncotator to annotate VCF file generated by strelka, there is ERROR message produced:

    anaconda2/lib/python2.7/site-packages/Oncotator-v1.8.0.0-py2.7.egg/oncotator/input/VcfInputMutationCreator.py", line 421, in _parseContigsMetadata
    keys = self.vcf_reader.contigs.keys()
    AttributeError: 'Reader' object has no attribute 'contains'

    Any ideas on how to fix this issues? the PyVCF version is 0.6.3.

    Thanks.

    Best,
    Xiangchun

  • rohan1925rohan1925 New YorkMember Posts: 5

    While running this : python setup.py install
    I am getting this error : error: Installed distribution numpy 1.4.1 conflicts with requirement numpy>=1.7.0

  • GeminghuiGeminghui Member Posts: 6

    Exactly the same error with oncotator 1.8.0.0 and python 2.7.12.
    Anyone could help?

    @arsala said:
    Hello

    I have downloaded and installed oncotator as well as all the dependencies. The installation of oncotator has been confirmed by running the command "oncotator -h" to invoke usage instructions.

    Now I am doing test run of oncotator with the following command: $ oncotator -v --db-dir /path/to/oncotator_v1_ds_Jan262015 test/testdata/maflite/Patient0.snp.maf.txt exampleOutput.tsv hg19

    and I am getting this error: TypeError: _open() got an unexpected keyword argument 'encoding'

    Please help me out.

  • jowkarjowkar GothenburgMember Posts: 1

    I get the exact same error as the previous poster, using 1.9.1.0 and running the test command:

    oncotator -v --db-dir ~/bin/oncotator_v1_ds_Jan262014 ~/bin/oncotator-1.9.1.0/test/testdata/maflite/Patient0.snp.maf.txt exampleOutput.tsv hg19
    
    Traceback (most recent call last):
      File "/home/joakim/.local/bin/oncotator", line 9, in <module>
        load_entry_point('Oncotator==v1.9.1.0', 'console_scripts', 'oncotator')()
      File "/home/joakim/.local/lib/python2.7/site-packages/Oncotator-v1.9.1.0-py2.7.egg/oncotator/Oncotator.py", line 309, in main
        annotator.annotate()
      File "/home/joakim/.local/lib/python2.7/site-packages/Oncotator-v1.9.1.0-py2.7.egg/oncotator/Annotator.py", line 437, in annotate
        filename = self._outputRenderer.renderMutations(mutations, metadata=metadata, comments=comments)
      File "/home/joakim/.local/lib/python2.7/site-packages/Oncotator-v1.9.1.0-py2.7.egg/oncotator/output/TcgaMafOutputRenderer.py", line 268, in renderMutations
        m = mutations.next()
      File "/home/joakim/.local/lib/python2.7/site-packages/Oncotator-v1.9.1.0-py2.7.egg/oncotator/Annotator.py", line 448, in _applyManualAnnotations
        for m in mutations:
      File "/home/joakim/.local/lib/python2.7/site-packages/Oncotator-v1.9.1.0-py2.7.egg/oncotator/Annotator.py", line 456, in _applyDefaultAnnotations
        for m in mutations:
      File "/home/joakim/.local/lib/python2.7/site-packages/Oncotator-v1.9.1.0-py2.7.egg/oncotator/Annotator.py", line 519, in _annotate_mutations_using_datasources
        m = self._annotate_func_ptr(m, datasource)
      File "/home/joakim/.local/lib/python2.7/site-packages/Oncotator-v1.9.1.0-py2.7.egg/oncotator/Annotator.py", line 88, in _annotate_mut
        return ds.annotate_mutation(m)
      File "/home/joakim/.local/lib/python2.7/site-packages/Oncotator-v1.9.1.0-py2.7.egg/oncotator/datasources/TabixIndexedVcfDatasource.py", line 230, in annotate_mutation
        vcf_records = self.vcf_reader.fetch(mutation.chr, mut_start - 1, mut_end)  # query database for records
      File "/usr/local/lib/python2.7/site-packages/PyVCF-0.6.8-py2.7.egg/vcf/parser.py", line 626, in fetch
        encoding=self.encoding)
      File "ctabix.pyx", line 92, in pysam.ctabix.Tabixfile.__cinit__ (lib/pysam/ctabix.c:2234)
      File "ctabix.pyx", line 98, in pysam.ctabix.Tabixfile._open (lib/pysam/ctabix.c:2344)
    TypeError: _open() got an unexpected keyword argument 'encoding'
    
Sign In or Register to comment.