If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We appreciate your help!
Test-drive the GATK tools and Best Practices pipelines on Terra
Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
(howto) Install and run Oncotator for the first time
Geraldine_VdAuwera
Cambridge, MAMember, Administrator, Broadie admin
1. Download the Oncotator package, the default datasources package, and (recommended) transcript override list from the Downloads page
Please note: Broadies who wish to run the installed Oncotator on the Broad cluster should follow the instructions here, instead of this page
Oncotator Download
- Download the latest release here.
Default Datasource Corpus Download (April 5, 2016)
- Download 17GB oncotator_v1_ds_April052016.tar.gz.
Please note that this corpus should be used with Oncotator 1.4.x.x and above. Uniprot AA Pos annotations will not function properly with Oncotator 1.3.x.x and below.
Transcript override lists
We highly recommend that you download and use one of the below transcript override lists, especially if clinical applications of Oncotator. When running Oncotator, provide one of the below files with the -c parameter.
Download UniProt Exact Match For GENCODE v19, will give selection priority to transcripts with protein sequences that match the UniProt protein sequence exactly. This file can also be found in the Oncotator download at
test/testdata/tx_exact_uniprot_matches.txt.Download UniProt Exact Match + Clinical For GENCODE v19, this will give priority to known clinical protein changes. This file is a modification of the UniProt Exact Match (above). For more information about how this list was generated, please see the powerpoint presentation here
The Oncotator and default datasource corpus packages are simple tar files that can be expanded using the following commands:
$ tar zxvf oncotator-1.5.1.0.tar.gz
$ tar zxvf oncotator_v1_ds_Jan262015.tar.gz
This will produce two directories called oncotator-1.5.1.0 and oncotator_v1_ds_Dec112014, respectively. Move to the oncotator-1.5.1.0 directory by doing:
$ cd oncotator-1.5.1.0
2. Set up your Python environment and install dependencies
See the article on platform requirements for a full list of dependencies. This tutorial will show you how to use the virtual environment script we provide to set everything up automagically, and this tutorial will show you how to install dependencies manually if needed (or preferred).
3. Install Oncotator
Once you have installed all the necessary dependencies listed above, simply run the standard Python install script which is included with the Oncotator distribution.
$ python setup.py install
Two binaries (executable program files) named oncotator and initializeDatasource respectively will be installed into your Python's bin/ directory. You can test that they were installed by running e.g.:
$ oncotator -h
to invoke the help / usage instructions. You can also do a test run of Oncotator on the Patient0.snp.maf.txt file provided with the Oncotator distribution (in the test/testdata/maflite/ directory) with the following command:
$ oncotator -v --db-dir /path/to/oncotator_v1_ds_Jan262015 test/testdata/maflite/Patient0.snp.maf.txt exampleOutput.tsv hg19
where you provide the location of the datasources using the --db-dir argument. You may need to adapt the file path for the Patient0.snp.maf.txt file depending on where you run this command from.
This will produce a new file named exampleOutput.tsv with the appropriate annotations, built against the hg19 reference.
Comments
An error occurred while installing Oncotator.
error: Setup script exited with error: command 'gcc' failed with exit status 1
What do i do ?
@pmint, have you looked at the compiler requirements in this document?
An error occurred while installing Oncotator.
Yachtmaster
@szhwatch, we're going to need a little more information than that to help you. What did you try, what happened, ...?
Hi Geraldine,
Thanks you and your team at Broad for providing a local installation of oncotator. Could you provide the checksum for the datasource corpus ? I want to ensure that the download went perfectly.
Thanks!
Cheers.
Good day,
I was running the test set and a got a series of warnings :
Could you please advice if this is expected?
Cheers
Have a look at this FAQ: http://gatkforums.broadinstitute.org/discussion/4350/what-does-this-error-mean-entrez-gene-id-was-zero-but-hugo-symbol-was-not-unknown#latest
Hi Geraldine,
I have the same issue. The link specified above is broken. Would you please provide a new one for me? I really appreciate it.
Best,
Joon
http://gatkforums.broadinstitute.org/discussion/4157/what-does-an-error-message-mean#latest
I was install oncotator and got a error: ERROR: Could not load pysam. Some features will be disabled (e.g. COSMIC annotations) and may cause Oncotator to fail. No handlers could be found for logger "root". I used the virtual environment script and the pysam-0.7.5 package have been successfully installed. I was confused why got this error when I test the oncotator. Can you give me some suggestion?
Hi @vguest,
Can you post the output that was generated during the installation process?
We love having the local installation of Oncotator because of the improved ease of use and convenience. We do trip up ourselves from time to time when we forgot whether the dozens of output files contain canonical or best effect annotations or that the --tx-mode flag was set to "EFFECT" or not. Can you add as a future enhancement a text string in the output file to label the output as either Canonical or Best Effect? The text string does not have to be in the column headers but wherever it is sensible for you to put it.
Hi @chung2000, we've put this in as a feature request. In the meantime, one way to keep track of this is to check the run logs.
Thank you @Geraldine_VdAuwera. While the run logs are excellent records for when, where, and how the --tx-mode flag was set, all is lost when the Oncotator output files gets passed around to other people in our group or to other organizations because we rather not send the run logs to those people.
@chung2000 I understand, it is awkward to rely on log files. This will be implemented in an upcoming version -- we'll make an announcement when the next release is ready.
If I want to obtain the latest version of the datasource corpus (more recent than the June 2014 release) where is it located? Thank you.
@chung2000 I don't believe there is a later version than June 2014 as of yet. When there is it will be made available on this page.
Hi, when I run oncotator, I keep getting the error "pkg_resources.DistributionNotFound: natsort"
I am new to command-line, but I have checked that the natsort package is present. Any help would be greaty appreciated. Thanks
@np3 Try "pip install -U distribute" That has fixed this in the past.
Thanks for the suggestion. Unfortunately, after doing this step and trying again, I received the same error message.
We are using Ubuntu 12.04 LTS. Would that make a difference?
@np3 Can you post the results of "pip freeze" ?
@np3 I have been using Ubuntu 12.04 with no issues. Also, what version of python are you using?
Python==2.7.3
Brlapi==0.5.6
Cython==0.21
GnuPGInterface==0.3.2
Jinja2==2.7.1
Mako==0.5.0
MarkupSafe==0.15
MySQL-python==1.2.3
PAM==0.4.2
PIL==1.1.7
Pillow==2.3.0
Pyste==0.9.10
Twisted-Core==11.1.0
Twisted-Names==11.1.0
Twisted-Web==11.1.0
UniConvertor==1.1.4
adium-theme-ubuntu==0.3.2
apt-xapian-index==0.44
apturl==0.5.1ubuntu3
argparse==1.2.1
biopython==1.62
chardet==2.0.1
chimerascan==0.4.5
classicmenu-indicator==0.09
command-not-found==0.2.44
configglue==1.0
configobj==4.7.2
debtagshw==0.1
decorator==3.3.2
defer==1.0.2
dirspec==3.0.0
duplicity==0.6.18
httplib2==0.7.2
ipython==0.12.1
jockey==0.9.7
keyring==0.9.2
language-selector==0.1
launchpadlib==1.9.12
lazr.restfulclient==0.12.0
lazr.uri==1.0.3
lockfile==0.8
louis==2.3.0
lxml==2.3.2
matplotlib==1.1.1rc
natsort==3.5.1
nose==1.1.2
numexpr==1.4.2
numpy==1.6.1
nvidia-common==0.0.0
oauth==1.0.1
onboard==0.97.1
oneconf==0.2.8.1
pandas==0.7.0
pdfshuffler==0.6.0
pexpect==2.3
piston-mini-client==0.7.2
protobuf==2.4.1
pyOpenSSL==0.12
pyPdf==1.13
pycrypto==2.4.1
pycups==1.9.61
pycurl==7.19.0
pyinotify==0.9.2
pyparsing==1.5.2
pysam==0.7.5
pyserial==2.5
pysmbc==1.0.13
pysqlite==1.0.1
python-apt==0.8.3ubuntu7.2
python-dateutil==1.5
python-debian==0.1.21ubuntu1
python-virtkey==0.60.0
pytz==2011k
pyxdg==0.19
pyzmq==2.1.11
reportlab==2.5
rhythmbox-ubuntuone==4.2.0
rpy==1.0.3
scikits.statsmodels==0.3.1
scipy==0.9.0
sessioninstaller==0.0.0
simplegeneric==0.7
simplejson==2.3.2
software-center-aptd-plugins==0.0.0
sympy==0.7.1.rc1
system-service==0.1.6
tables==2.3.1
tornado==2.1
ubuntuone-couch==0.3.0
ubuntuone-installer==3.0.2
ubuntuone-storage-protocol==3.0.2
ufw==0.31.1-1
unattended-upgrades==0.1
unity-lens-video==0.3.5
unity-scope-video-remote==0.3.5
urlgrabber==3.9.1
usb-creator==0.2.23
virtualenv==1.11.6
wadllib==1.3.0
wsgiref==0.1.2
xdiagnose==2.5.3
xkit==0.0.0
xlrd==0.9.2
yum-metadata-parser==1.1.2
zope.interface==3.6.1
@chung2000 @Geraldine_VdAuwera I have posted the Sept 17 corpus here.
@np3 Though I have newer packages that is very similar to my configuration. Usually the issue here is an incompatible version of distribute or setuptools with the version of python. Are you using a virtual environment? In other words, did you run
scripts/create_oncotator_venv.sh? Apologies for basic questions. Also, use @LeeTL1220 to get a faster response.@np3 For a different project, I just got the same error as you and I just ran
pip uninstall distributeand it worked.Hello, I have problems making oncotator work, the first one is pysam 0.7.5, I can install 0.7.8 but for 0.7.5 I get:
NameError: name 'sys_platform' is not defined
/tmp/pip_build_rbarrant/pysam/distribute-0.6.34-py2.7.egg
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_rbarrant/pysam/setup.py", line 131, in
File "distribute_setup.py", line 152, in use_setuptools
File "distribute_setup.py", line 132, in _do_download
File "distribute_setup.py", line 123, in _build_egg
IOError: Could not build the egg.
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_rbarrant/pysam
Storing debug log for failure in /users/r/b/rbarrant/.pip/pip.log
bash-3.2$
Can things work with 0.7.8 or do I REALLY need 0.7.5?
Thanks in advance,
Ramiro
@LeeTL1220 Thanks for the info. I previously used "pip install virtualenv". I now tried your suggestion for installing the VE. I also uninstalled distribute before and after doing so, and also reinstalled it at the end. I could not get oncotator to work at any of these steps. I should say I am very new to Linux, so any basic advice is appreciated.
Btw, I used the command "./oncotator -h" from the "bin" directory. Is this correct?
The next step would be to try to use the new Corpus. To do so, do I need to uninstall/remove the old one?
Thanks for your help.
@np3, did you try running through the installation tutorial provided here? : http://gatkforums.broadinstitute.org/discussion/4162/howto-set-up-a-virtual-environment-and-install-all-dependencies-for-oncotator
@Geraldine_VdAuwera Sorry for the late reply. Yes, that is guide I used. Thanks.
@np3, if your installation is like mine, you would use the the command "./oncotator -h" from the "oncotator-1.3.0.0" directory.
Also, you don't need to remove the old Corpus unless you want to recover the disk space that it is occupying.
Hello,
I have an error while using VCF file : ValueError: could not convert string to float:
I have install the patched version of PyVCF.
Regards
@JeremyS Did you use the
create_oncotator_venv.shscript to create a virtual environment and run oncotator from there?Having spent a few days making Oncotator to work, I want to share my solution. I think the main problem stems from the fact that Oncotator uses distribute-0.6.15 which does not understand wheel format (new Python packaging). Therefore if you happen to have some of the required packages installed as dist-info rather than egg-info you are likely to run into trouble (check your lib/python2.7/site-packages/ )
We are running SL 6.4 on our cluster so no Python 2.7 in the distro and we are using modules.
1/ installed python/2.7.9; created a module for python/2.7.9
2/ installed numpy and scipy with Intel MKL support; after that
This is a minimal install. Uninstall nose before proceeding further (nose was needed for testing numpy)
After that you could proceed with virtualenv setup as recommended elsewhere. But we prefer modules.
3/ created oncotator module containing this
4/ downloaded and unpacked oncotator 1.4.1.0
5/ adapted oncotator-1.4.1.0/scripts/create_oncotator_venv.sh to make my own script for installing prerequisites
cython and numpy are removed from C_PACKAGE_LIST because we have them already. Should not be any harm if there are not removed.
6/ installed oncotator itself
7/ after that loading
should make oncotator available on the command line.
8/ it should be also fine to do the following
i.e. install nose back into the new python because nose is such an essential package. Oncotator should still be happy with its nose egg install.
@ink, thanks for reporting your solution.
I unzipped oncotator_v1_ds_Jan262015.tar.gz and found that the folder name was oncotator_v1_ds_Jan262014 instead of oncotator_v1_ds_Jan262015. Was it simply a typo?
@woodydon said:
I found the same thing but was able to run Oncotator with the new corpus to get the upgraded 1000 genome and dbDNP data. Most likely the folder name had a typo, as is usually the case at the beginning of any year.
Yes that looks like a typo, sorry about that.
Hello, I've installed oncotator/1.5.1.0 and try to run it with python/2.7.3 on my vcf files with the following command line:
oncotator -i VCF --db-dir $TMPDIR/oncotator_db -o VCF $TMPDIR/merged.vcf $TMPDIR/annotated.vcf hg19
I keep getting the following error message:
2015-04-28 20:17:21,924 ERROR [oncotator.output.VcfOutputRenderer:222] Traceback (most recent call last):
File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/VcfOutputRenderer.py", line 214, in _renderSortedTsv
recordBuilder = self._parseRecordBuilder(m, recordBuilder, dataManager, inferGenotypes)
File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/VcfOutputRenderer.py", line 276, in _parseRecordBuilder
recordBuilder.addInfo(sampleName, ID, num, dataType, val, isSplit)
File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/RecordBuilder.py", line 291, in addInfo
val = self._map(MutUtils.str2bool, self._fixVal(val, isSplit)) # convert the value to boolean
File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/output/RecordBuilder.py", line 97, in _map
return [func(v) if v not in bad else None for v in iterable]
File "/apps/python/2.7.3/lib/python2.7/site-packages/Oncotator-v1.2.7.0-py2.7.egg/oncotator/utils/MutUtils.py", line 222, in str2bool
return v.lower() in ("yes", "true", "t", "1", "y")
AttributeError: 'NoneType' object has no attribute 'lower'
2015-04-28 20:17:21,924 ERROR [oncotator.output.VcfOutputRenderer:223] Error at mutation 1 ['1', '19865', '19865']:
Can you please help with this issue?
Thanks in advance,
Alona
I recently downloaded Oncotator in order to use it on open-access TCGA MAF files. I kept only the first 34 columns of the MAF file, in order to remove any additional columns that may have been added by one of the TCGA data centers, and tried to input it to Oncotator using the --input_format=MAFLITE option. It seems like the two "dbSNP" columns in the TCGA MAF format caused a problem, and I got this error:
Traceback (most recent call last):
File "/tools/bin/Oncotator", line 9, in
load_entry_point('Oncotator==v1.5.1.0', 'console_scripts', 'Oncotator')()
File "/tools/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Oncotator.py", line 292, in main
annotator.annotate()
File "/tools/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 403, in annotate
filename = self._outputRenderer.renderMutations(mutations, metadata=metadata, comments=comments)
File "/tools/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/output/TcgaMafOutputRenderer.py", line 283, in renderMutations
raise e
oncotator.DuplicateAnnotationException.DuplicateAnnotationException: 'Attempting to create an annotation multiple times (dbSNP_RS) with old, new values of (, rs377395363)'
Removing the dbSNP_RS and dbSNP_Val_Status columns eliminated this problem.
Another issue I noticed is that column 12 in the annotated output ("Tumor_Seq_Allele1") is actually a copy of the preceding column ("Reference_Allele"), while column 13 ("Tumor_Seq_Allele2") is almost always the same as the input column 13, but sometimes different. Also the NCBI_Build in column #4 gets reset to "UNKNOWN" (although column #196 has the correct build information).
I was expecting that columns 1-34 in the original MAF would be the same in the output annotated MAF -- is there a reason why this is not the case?
thanks,
Sheila
If you upgrade to Oncotator 1.5.3.0, I believe this issue has been fixed.
@sherey65 Yes, removing the dbSNP annotations in the input file should eliminate the issue. As per the TCGA MAF spec, Reference_Allele and Tumor_Seq_Allele1 should be the same. If Tumor_Seq_Allele1 is the same as Tumor_Seq_Allele2, then you should check your input file. There are specific instructions for your use case (TCGA MAF in and TCGA MAF out) on the following page: http://gatkforums.broadinstitute.org/discussion/4177/what-input-files-can-i-annotate-with-oncotator#latest
Running the test file throws this error:
2015-05-22 05:52:49,823 INFO [oncotator.Oncotator:235] Oncotator v1.5.1.0
2015-05-22 05:52:49,823 INFO [oncotator.Oncotator:236] Args: Namespace(cache_url=None, canonical_tx_file=None, dbDir='oncotator_db', default_cli=[], default_config=None, genome_build='hg19', infer_genotypes='false', infer_onps=False, input_file='Patient0.snp.maf.txt', input_format='MAFLITE', log_name='oncotator.log', noMulticore=False, output_file='myOutputFile.maf.annotated', output_format='TCGAMAF', override_cli=[], override_config=None, prepend=False, read_only_cache=False, skip_no_alt=False, tx_mode='CANONICAL', verbose=6)
2015-05-22 05:52:49,823 INFO [oncotator.Oncotator:237] Log file: /mergedcancerdata/oncotator.log
2015-05-22 05:52:49,823 WARNING [oncotator.Oncotator:243] ngslib module not installed. Will be unable to annotate with BigWig datasources.
2015-05-22 05:52:49,824 INFO [oncotator.input.MafliteInputMutationCreator:115] Initializing a maflite file with the following header: ['build', 'chr', 'start', 'end', 'ref_allele', 'tum_allele1', 'tum_allele2', 'tumor_barcode', 'normal_barcode', 't_lod_fstar', 't_alt_count', 't_ref_count']
2015-05-22 05:52:49,825 INFO [oncotator.output.TcgaMafOutputRenderer:90] Building alternative keys dictionary...
2015-05-22 05:52:49,825 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for oncotator_db/1000genome_db/hg19/1000genome_db.config
2015-05-22 05:52:49,825 INFO [oncotator.DatasourceFactory:304] Creating pool
2015-05-22 05:52:49,835 INFO [oncotator.DatasourceFactory:307] Pool created
Traceback (most recent call last):
File "/usr/local/bin/oncotator", line 9, in
load_entry_point('Oncotator==v1.5.1.0', 'console_scripts', 'oncotator')()
File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Oncotator.py", line 288, in main
other_opts=determineOtherOptions(args), annotating_type=annotating_type)
File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/utils/RunSpecificationFactory.py", line 118, in create_run_spec
datasource_list = DatasourceFactory.createDatasources(datasourceDir, genomeBuild, isMulticore=isMulticore, numCores=numCores, tx_mode=tx_mode)
File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 294, in createDatasources
result = DatasourceFactory._createDatasourcesMulticore(numCores, dsQueueList)
File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 317, in _createDatasourcesMulticore
result.append(DatasourceFactory.createDatasourceGivenTuple(dsTuple))
File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 106, in createDatasourceGivenTuple
return DatasourceFactory.createDatasource(configTuple[0], configTuple[1])
File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 100, in createDatasource
return DatasourceFactory.createDatasourceFromConfigParser(configParser, leafDir)
File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/DatasourceFactory.py", line 186, in createDatasourceFromConfigParser
match_mode=configParser.get('general', 'match_mode'))
File "/usr/local/lib/python2.7/dist-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/datasources/TabixIndexedVcfDatasource.py", line 84, in init
self.vcf_reader = vcf.Reader(filename=src_file, strict_whitespace=True)
File "/usr/local/lib/python2.7/dist-packages/PyVCF-0.6.7-py2.7-linux-x86_64.egg/vcf/parser.py", line 288, in init
self._parse_metainfo()
File "/usr/local/lib/python2.7/dist-packages/PyVCF-0.6.7-py2.7-linux-x86_64.egg/vcf/parser.py", line 304, in _parse_metainfo
line = self.reader.next()
File "/usr/local/lib/python2.7/dist-packages/PyVCF-0.6.7-py2.7-linux-x86_64.egg/vcf/parser.py", line 268, in
self.reader = (line.strip() for line in self._reader if line.strip())
File "/usr/lib/python2.7/gzip.py", line 451, in readline
c = self.read(readsize)
File "/usr/lib/python2.7/gzip.py", line 258, in read
if not self._read(readsize):
File "/usr/lib/python2.7/gzip.py", line 293, in _read
self._read_gzip_header()
File "/usr/lib/python2.7/gzip.py", line 205, in _read_gzip_header
self._read_exact(struct.unpack("<H", self._read_exact(2)))
File "/usr/lib/python2.7/gzip.py", line 185, in _read_exact
data = self.fileobj.read(n)
TypeError: an integer is required
Hello!
I recently downloaded oncotator v1.5.3.0 and the newest database bundle (Jan262014) and I cannot annotate my samples. Oncotator works well and parses through my input file, but there is no annotations at the end. Apparently the datasources are not recognized:
2015-05-26 12:34:23,695 INFO [oncotator.Oncotator:235] Oncotator v1.5.3.0
2015-05-26 12:34:23,696 INFO [oncotator.Oncotator:236] Args: Namespace(cache_url=None, canonical_tx_file=None, dbDir='/data2/unige/dmg/xbonilla/oncotator_v1_ds_Jan262015/', default_cli=[], default_config=None, genome_build='hg19', infer_genotypes='false', infer_onps=True, input_file='call_stats_5-VS018-T1.out', input_format='MAFLITE', log_name='5-VS018-T1.ann.log', noMulticore=False, output_file='5-VS018-T1.maf.annotated', output_format='TCGAMAF', override_cli=[], override_config=None, prepend=False, read_only_cache=False, skip_no_alt=False, tx_mode='CANONICAL', verbose=6)
2015-05-26 12:34:23,696 WARNING [oncotator.Oncotator:243] ngslib module not installed. Will be unable to annotate with BigWig datasources.
2015-05-26 12:34:23,696 INFO [oncotator.utils.RunSpecificationFactory:108] Output file order may be different than input file because we're combining SNPs into ONPs
2015-05-26 12:34:34,261 INFO [oncotator.input.MafliteInputMutationCreator:115] Initializing a maflite file with the following header: ['contig', 'position', 'context', 'ref_allele', 'alt_allele', 'tumor_name', 'normal_name', 'score', 'dbsnp_site', 'covered', 'power', 'tumor_power', 'normal_power', 'total_pairs', 'improper_pairs', 'map_Q0_reads', 't_lod_fstar', 'tumor_f', 'contaminant_fraction', 'contaminant_lod', 't_ref_count', 't_alt_count', 't_ref_sum', 't_alt_sum', 't_ref_max_mapq', 't_alt_max_mapq', 't_ins_count', 't_del_count', 'normal_best_gt', 'init_n_lod', 'n_ref_count', 'n_alt_count', 'n_ref_sum', 'n_alt_sum', 'judgement']
2015-05-26 12:34:34,565 INFO [oncotator.input.OnpCombiner:77] Merging adjacent snps from the same samples into ONPs
2015-05-26 12:34:34,582 INFO [oncotator.output.TcgaMafOutputRenderer:90] Building alternative keys dictionary...
2015-05-26 12:34:34,584 WARNING [oncotator.DatasourceFactory:260] %s does not exist, so there will be no datasources.
2015-05-26 12:34:34,584 INFO [oncotator.DatasourceFactory:325] No datasources to initialize
2015-05-26 12:34:34,584 INFO [oncotator.cache.DummyCache:57] No cache specified. All cache attempts will be listed as cache misses.
2015-05-26 12:34:34,585 INFO [oncotator.Annotator:392] Annotating with 0 datasources: Oncotator v1.5.3.0 |
2015-05-26 12:34:34,585 INFO [oncotator.output.TcgaMafOutputRenderer:218] TCGA MAF output file: newmutect_newdatabase_5-VS018-T1.maf.annotated
2015-05-26 12:34:34,586 INFO [oncotator.output.TcgaMafOutputRenderer:219] Render starting...
2015-05-26 12:34:34,586 WARNING [oncotator.Annotator:466] THERE ARE NO DATASOURCES REGISTERED
2015-05-26 12:34:34,588 INFO [oncotator.utils.SampleNameSelector:90] Sample name is the concatenation of the normal_name and tumor_name columns.
2015-05-26 12:34:34,955 WARNING [oncotator.input.OnpQueue:196] Mutations are not sorted by start position, this may cause unexpected behavior or increased memory requirements. It is recommended that your sort any files that youusing with --infer-onps by position and sample name.
2015-05-26 12:34:35,685 INFO [oncotator.output.TcgaMafOutputRenderer:276] Rendered 1000 mutations.
...
I was wondering if I have to initialize one by one the databases from the bundle, or if there is a way of getting oncotator to recognize them since they are already in the right format? I am not sure what I missed in the installation process.
Thank you in advance and I'm sorry for the newbie question!
@mwong I think v1.5.3.0 fixes this issue.
@XBon I would need to see your directory structure, but I think you need to add one more dir to the end of your db-dir. db-dir should specify the parent directory of ref_hg/, hgnc/, dbnsfp/, etc.
I ran oncotator1.5.1.0 on a HC called VCF using the command
oncotator -i VCF -o VCF --db-dir /path_to/ngstools/oncotator_v1_ds_Jan262014/ Project_bwa_GatkHC.vcf Project_bwa_GatkHC_oncotator.vcf hg19
The script ran for about 214000 mutations and threw an exception. Pasting the error below. Can someone help me understand this error?
2015-06-11 13:11:58,655 INFO [oncotator.output.OutputDataManager:194] Wrote 214000 mutations to tsv.
Traceback (most recent call last):
File "/home/Pythonmodules/bin/oncotator", line 9, in
load_entry_point('Oncotator==v1.5.1.0', 'console_scripts', 'oncotator')()
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Oncotator.py", line 292, in main
annotator.annotate()
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 403, in annotate
filename = self.outputRenderer.renderMutations(mutations, metadata=metadata, comments=comments)
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/output/VcfOutputRenderer.py", line 118, in renderMutations
dataManager = OutputDataManager(self.configTable, mutations, comments, metadata, path)
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/output/OutputDataManager.py", line 92, in __init_
self.chroms, self.sampleNames, self.filename = self._writeMuts2Tsv(self.mutations, path)
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/output/OutputDataManager.py", line 160, in _writeMuts2Tsv
for mut in muts:
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 414, in _applyManualAnnotations
for m in mutations:
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 422, in _applyDefaultAnnotations
for m in mutations:
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/Annotator.py", line 470, in _annotate_mutations_using_datasources
for m in mutations:
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/input/VcfInputMutationCreator.py", line 306, in createMutations
sampleMut = self._addGenotypeData2Mutation(sampleMut, record, index)
File "/home/Pythonmodules/lib/python2.7/site-packages/Oncotator-v1.5.1.0-py2.7.egg/oncotator/input/VcfInputMutationCreator.py", line 130, in _addGenotypeData2Mutation
val = genotypeData[ID][index]
IndexError: list index out of range
Hello,
I am having the same issue as @mwong. I tried both versions 1.5.1.0 and 1.5.3.0 and it stops on the error "TypeError: an integer is required" as soon as it starts. Any ideas how to fix this?
Alessandro
=====
2015-06-17 16:15:01,636 INFO [oncotator.Oncotator:237] Log file: /data1/alessandro/projects/tools/oncotator-1.5.3.0/oncotator.log
2015-06-17 16:15:01,636 INFO [oncotator.utils.RunSpecificationFactory:108] no prepend specified when output is not TCGAMAF. Ignoring and proceeding.
2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/ccle_by_gp/hg19/ccle_by_gp.config
2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/tcgascape/hg19/tcgascape.config
2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/esp6500SI_v2_coverage/hg19/esp6500SI_v2_coverage.config
2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/esp6500SI_v2/hg19/esp6500SI_v2.config
2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/achilles/hg19/achilles.config
2015-06-17 16:15:01,646 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/oreganno/hg19/oreganno.config
2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/hgnc/hg19/hgnc.config
2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/ccle_by_gene/hg19/ccle_by_gene.config
2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/clinvar/hg19/clinvar.config
2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/dna_repair_genes/hg19/dna_repair_genes.config
2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/cancer_gene_census/hg19/cancer_gene_census.config
2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/dbsnp/hg19/dbsnp.config
2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/cosmic_tissue/hg19/cosmic_tissue.config
2015-06-17 16:15:01,647 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/gencode_out2/hg19/gencode_out2.config
2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/uniprot_aa_annotation/hg19/uniprot_aa_annotation.config
2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/ref_hg/hg19/ref_hg.config
2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/cosmic_fusion/hg19/cosmic_fusion.config
2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/uniprot_aa_xform/hg19/uniprot_aa_xform.config
2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/tumorscape/hg19/tumorscape.config
2015-06-17 16:15:01,648 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/mutsig/hg19/mutsig.config
2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/cosmic/hg19/cosmic.config
2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/1000genome_db/hg19/1000genome_db.config
2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/dbNSFP_ds/hg19/dbNSFP_ds.config
2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/familial/hg19/familial.config
2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/gencode_xrefseq/hg19/gencode_xrefseq.config
2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/simple_uniprot/hg19/simple_uniprot.config
2015-06-17 16:15:01,649 INFO [oncotator.DatasourceFactory:286] Queuing datasource creation for /home/alessandro/projects/tools/oncotator_v1_ds_Jan262014/so_terms/hg19/so_terms.config
2015-06-17 16:15:01,650 INFO [oncotator.DatasourceFactory:304] Creating pool
2015-06-17 16:15:01,658 INFO [oncotator.DatasourceFactory:307] Pool created
Traceback (most recent call last):
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/bin/oncotator", line 9, in
load_entry_point('Oncotator==1.5.3.0', 'console_scripts', 'oncotator')()
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/Oncotator.py", line 288, in main
other_opts=determineOtherOptions(args), annotating_type=annotating_type)
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/utils/RunSpecificationFactory.py", line 118, in create_run_spec
datasource_list = DatasourceFactory.createDatasources(datasourceDir, genomeBuild, isMulticore=isMulticore, numCores=numCores, tx_mode=tx_mode)
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 294, in createDatasources
result = DatasourceFactory._createDatasourcesMulticore(numCores, dsQueueList)
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 317, in _createDatasourcesMulticore
result.append(DatasourceFactory.createDatasourceGivenTuple(dsTuple))
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 106, in createDatasourceGivenTuple
return DatasourceFactory.createDatasource(configTuple[0], configTuple[1])
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 100, in createDatasource
return DatasourceFactory.createDatasourceFromConfigParser(configParser, leafDir)
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/DatasourceFactory.py", line 186, in createDatasourceFromConfigParser
match_mode=configParser.get('general', 'match_mode'))
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/Oncotator-1.5.3.0-py2.7.egg/oncotator/datasources/TabixIndexedVcfDatasource.py", line 84, in init
self.vcf_reader = vcf.Reader(filename=src_file, strict_whitespace=True)
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/PyVCF-0.6.4-py2.7-linux-x86_64.egg/vcf/parser.py", line 283, in init
self._parse_metainfo()
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/PyVCF-0.6.4-py2.7-linux-x86_64.egg/vcf/parser.py", line 299, in _parse_metainfo
line = self.reader.next()
File "/data1/alessandro/projects/tools/oncotator-1.5.3.0/venv/local/lib/python2.7/site-packages/PyVCF-0.6.4-py2.7-linux-x86_64.egg/vcf/parser.py", line 263, in
self.reader = (line.strip() for line in self._reader if line.strip())
File "/usr/lib/python2.7/gzip.py", line 451, in readline
c = self.read(readsize)
File "/usr/lib/python2.7/gzip.py", line 258, in read
if not self._read(readsize):
File "/usr/lib/python2.7/gzip.py", line 293, in _read
self._read_gzip_header()
File "/usr/lib/python2.7/gzip.py", line 205, in _read_gzip_header
self._read_exact(struct.unpack("<H", self._read_exact(2)))
File "/usr/lib/python2.7/gzip.py", line 185, in _read_exact
data = self.fileobj.read(n)
TypeError: an integer is required
@alagana @mwong Did you guys use the oncotator create_oncotator_venv.sh script?
@LeeTL1220 I did. I followed the instructions on http://gatkforums.broadinstitute.org/discussion/4162/howto-set-up-a-virtual-environment-and-install-all-dependencies-for-oncotator
Hi guys, I downloaded the virtual environment successfully and was able to run the create_oncotator_venv.sh script but I am not able to run - python setup.py install. I tried this inside the virtual environment as well but it gave me the error -
** File "numpy/core/setup.py", line 42, in check_types
7.2 Termination. LICENSEE shall have the right to terminate this Agreement for any reason upon prior written notice to BROAD. If LICENSEE breaches any provision hereunder, and fails to cure such breach within thirty (30) days, BROAD may terminate this Agreement immediately. Upon termination, LICENSEE shall provide BROAD with written assurance that the original and all copies of the PROGRAM have been destroyed, except that, upon prior written authorization from BROAD, LICENSEE may retain a copy for archive purposes.
File "numpy/core/setup.py", line 293, in check_types
SystemError: Cannot compile 'Python.h'. Perhaps you need to install python-dev|python-devel.**
I also tried - apt-get install python-dev. I verified if I have Numpy installed in Python and I do have it. Please let me know how to fix this and run oncotator. I am not sure what I am missing here. Thank you!
Hi,
When running oncotator, I got the following error message like this: ImportError: No module named _sqlite3
I'm not good at python. Can your experts solve this problem?
And here's my command line:
oncotator -v --db-dir $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014 --input_format=VCF $HOME/K104416/GATK/K104416.output.filter.vcf Output.tsv hg19
And I'm sure sqlite version 3.08.11.1 had been loaded.
Never mind the above problem. Now it worked after I re-started my putty.
But here's another question. I don't know why no datasources to initialize. It still can render all mutations.
2015-09-13 17:52:47,220 WARNING [oncotator.Oncotator:243] ngslib module not installed. Will be unable to annotate with BigWig datasources.
2015-09-13 17:52:47,387 INFO [oncotator.output.TcgaMafOutputRenderer:90] Building alternative keys dictionary...
2015-09-13 17:52:47,389 WARNING [oncotator.DatasourceFactory:260] %s does not exist, so there will be no datasources.
2015-09-13 17:52:47,389 INFO [oncotator.DatasourceFactory:325] No datasources to initialize
2015-09-13 17:52:47,389 INFO [oncotator.cache.DummyCache:57] No cache specified. All cache attempts will be listed as cache misses.
2015-09-13 17:52:47,389 INFO [oncotator.Annotator:392] Annotating with 0 datasources: Oncotator v1.5.1.0 |
2015-09-13 17:52:47,433 INFO [oncotator.output.TcgaMafOutputRenderer:218] TCGA MAF output file: Output.tsv
2015-09-13 17:52:47,433 INFO [oncotator.output.TcgaMafOutputRenderer:219] Render starting...
2015-09-13 17:52:47,433 WARNING [oncotator.Annotator:466] THERE ARE NO DATASOURCES REGISTERED
2015-09-13 17:52:48,702 INFO [oncotator.output.TcgaMafOutputRenderer:276] Rendered 1000 mutations.
...
@varsha Did you install python-dev after your created the virtualenv?
@daih You should see the datasource directories in
$HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014. For example, the directory$HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014/hgnc/hg19should exist. Perhaps, you have an extra subdirectory?these are what I alrealdy have, is there anything else?
$ ls hgnc/hg19
hgnc.config hgnc_download_Sept172014.bin hgnc_download_Sept172014.tsv
@daih Can you provide
ls -la $HOME/GATKOncotatorDatasource/oncotator_v1_ds_Jan262014? It is claiming that this directory does not exist.total 808
drwxr-xr-x 29 daih faculty 4096 Sep 10 15:13 .
drwxr-xr-x 3 daih faculty 4096 Sep 10 14:24 ..
drwxr-xr-x 3 daih faculty 4096 Jan 23 2015 1000genome_db
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 achilles
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 cancer_gene_census
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 ccle_by_gene
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 ccle_by_gp
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 clinvar
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 cosmic
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 cosmic_fusion
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 cosmic_tissue
drwxr-xr-x 3 daih faculty 4096 Jun 9 2014 dbNSFP_ds
drwxr-xr-x 3 daih faculty 4096 Jan 23 2015 dbsnp
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 dna_repair_genes
drwxr-xr-x 3 daih faculty 4096 Feb 7 2014 esp6500SI_v2
drwxr-xr-x 3 daih faculty 4096 Feb 7 2014 esp6500SI_v2_coverage
-rw-r--r-- 1 daih faculty 701609 Sep 10 15:13 exampleOutput.tsv
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 familial
drwxr-xr-x 3 daih faculty 4096 Sep 10 15:36 gencode_out2
drwxr-xr-x 4 daih faculty 4096 Nov 25 2014 gencode_xrefseq
drwxr-xr-x 3 daih faculty 4096 Sep 17 2014 hgnc
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 mutsig
-rw-r--r-- 1 daih faculty 2305 Sep 10 15:13 oncotator.log
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 oreganno
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 ref_hg
drwxr-xr-x 3 daih faculty 4096 Dec 5 2014 simple_uniprot
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 so_terms
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 tcgascape
drwxr-xr-x 3 daih faculty 4096 Oct 22 2013 tumorscape
drwxr-xr-x 3 daih faculty 4096 Dec 5 2014 uniprot_aa_annotation
drwxr-xr-x 4 daih faculty 4096 Dec 15 2014 uniprot_aa_xform
Hi could you please send me the link to download the oncotator-1.5.3.0 version? Thank you.
@varsha Just updated to the latest.
Hi,
Is it possible to host oncotator with a web front end [like the one on Broad's website] locally ? I want to use the JSON API to annotate variants from other hosts. Thanks.
IS there a way I can manually pick the annotations I want and not all the tracks by default?
Hello,
I'm able to run Oncotator v1.5.1.0 successfully on many samples. I'm using database source oncotator_v1_ds_Jan262015.tar.gz
The input file is a VCF and I'm using SIMPLE_TSV format as output.
I'm not able to find the COSMIC ID for the variants ( if exists in COSMIC ). Am I missing something obvious?
Please advise. Thanks.
@miked Unfortunately, COSMIC IDs are not provided.
Unfortunately there are still issues installing Oncotator. The principal issue related to v 1.8.0.0 is that it requires pysam 0.7.5 which is not available from pypi. As a result oncotator setup gives an error about missing pysam. The dependencies script installs everything but pysam in fact needlessly upgrading to the latest pysam version even if you happened to have the right one in the first place.
Here is my recipe:
1/ download and install pysam 0.7.5
https://pysam.googlecode.com/files/pysam-0.7.5.tar.gz
2/ install dependencies as listed in scripts/create_oncotator_venv.sh BUT change ngslib to "ngslib==1.1.9" where necessary because installing newer version will upgrade your pysam which you want to avoid
3/ install oncotator itself (python setup.py install)
Hope this helps
Hello
I have downloaded and installed oncotator as well as all the dependencies. The installation of oncotator has been confirmed by running the command "oncotator -h" to invoke usage instructions.
Now I am doing test run of oncotator with the following command: $ oncotator -v --db-dir /path/to/oncotator_v1_ds_Jan262015 test/testdata/maflite/Patient0.snp.maf.txt exampleOutput.tsv hg19
and I am getting this error: TypeError: _open() got an unexpected keyword argument 'encoding'
Please help me out.
@ramiro2k ,@ink
I confronted with same problem. I just modify the setup.py with editing 'pysam==0.7.5' to 'pysam' , pyvcf >= 0.6.8' to 'pyvcf == 0.6.8'
ps:The version in Github might fix some underlying bugs (https://github.com/broadinstitute/oncotator).
@arsala I have the exactly same problem, some one can help me out ?
@arsala @dtcdtcdtcdtc Can you guys run with the latest version (1.9.1.0) and the latest default datasources? Also, are you running python 2.7?
When I used oncotator to annotate VCF file generated by strelka, there is ERROR message produced:
anaconda2/lib/python2.7/site-packages/Oncotator-v1.8.0.0-py2.7.egg/oncotator/input/VcfInputMutationCreator.py", line 421, in _parseContigsMetadata
keys = self.vcf_reader.contigs.keys()
AttributeError: 'Reader' object has no attribute 'contains'
Any ideas on how to fix this issues? the PyVCF version is 0.6.3.
Thanks.
Best,
Xiangchun
While running this : python setup.py install
I am getting this error : error: Installed distribution numpy 1.4.1 conflicts with requirement numpy>=1.7.0
Exactly the same error with oncotator 1.8.0.0 and python 2.7.12.
Anyone could help?
I get the exact same error as the previous poster, using 1.9.1.0 and running the test command:
Hi,
Can I use Oncotator for mm10?
Many thanks, Rahel
Hi there,
I'm having problems trying to download: oncotator_v1_ds_April052016.tar.gz file.
When I try to decompress it, I get the following error:
gzip: stdin: unexpected end of file tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting nowThe problem is the VCF file:
gzip: ALL.wgs.phase3_shapeit2_mvncall_integrated_v5.20130502.sites.vcf.gz: unexpected end of fileAnyone is experiencing the same problem? Any help?
Thank you in advance,
Hi,
It looks like the link to this file '17GB oncotator_v1_ds_April052016.tar.gz' is broken. Can you please fix?
Thanks!
-Elizabeth
I wish there were other mirrors to download the data. The limit of 25 users seen a bit small.
@raonyguimaraes,
Sorry about that. I hope you were able to download what you needed.
Hi.Can i use it to annotate dog cancer data?
Hi,
I am having issues installing Oncotator 1.9.3.0 on a Linux system running ubuntu 16.04. I setup a virtual environment using the following howto
When setting up the environment it install ngslib=1.1.18 and install its dependent pysam=0.12.0.1.
I see the following in the terminal:
Installing collected packages: pysam, ngslib
Running setup.py install for pysam ... done
Running setup.py install for ngslib ... done
Successfully installed ngslib-1.1.18 pysam-0.12.0.1
When I go to install Oncotator it searches for pysam=0.9.0 and tries to install it but it errors:
Searching for pysam==0.9.0
Reading https://pypi.python.org/simple/pysam/
Downloading https://pypi.python.org/packages/0f/37/fa514cb2163997551e95f63ec12f7bec7b640a601456f1d0ab3ab900c05f/pysam-0.9.0.tar.gz#md5=66913000ff50817bdcf56dfd3a52396f
Best match: pysam 0.9.0
Processing pysam-0.9.0.tar.gz
Writing /tmp/easy_install-Y_XTwi/pysam-0.9.0/setup.cfg
pysam: htslib configure options: --enable-libcurl
pysam: libcurl of builtin htslib has been enabled, adding shared libcurl and libcrypto
package init file 'htslib/init.py' not found (or not a regular file)
package init file 'htslib/htslib/init.py' not found (or not a regular file)
package init file 'samtools/init.py' not found (or not a regular file)
package init file 'bcftools/init.py' not found (or not a regular file)
package init file 'samtools/win32/init.py' not found (or not a regular file)
warning: no files found matching 'KNOWN_BUGS'
warning: no files found matching 'THANKS'
warning: no files found matching 'samtools/configure'
warning: no files found matching 'samtools/config.mk.in'
warning: no files found matching 'samtools/config.h.in'
warning: no files found matching 'tests/pysam_data'
warning: no files found matching 'tests/tabix_data'
gcc: error: unrecognized command line option ‘-R’
gcc: error: unrecognized command line option ‘-R’
gcc: error: unrecognized command line option ‘-R’
error: Setup script exited with error: command 'gcc' failed with exit status 1
To get around that I install pysam=0.9.0 in that same virtual environment and then launched the create_oncotator_venv.sh bash script. Here are the tools installed
pip freeze returns:
bcbio-gff==0.6.2
biopython==1.66
Cython==0.24
enum34==1.1.2
futures==3.1.1
more-itertools==2.2
natsort==4.0.4
ngslib==1.1.18
nose==1.3.7
numpy==1.11.0
Oncotator==1.9.3.0
pandas==0.18.0
parse==1.8.2
pysam==0.9.0
python-dateutil==2.6.1
python-memcached==1.57
pytz==2017.3
PyVCF==0.6.8
shove==0.6.6
six==1.11.0
SQLAlchemy==1.0.12
stuf==0.9.16
When I install oncotator it returns
Using /home/ryan/oncatator_virt_env/lib/python2.7/site-packages
Finished processing dependencies for Oncotator==1.9.3.0
ngslib must be installed manually on non-mac: pip install --no-binary :all: ngslib==1.1.18
ngslib is installed according to the pip freeze command but if I try to import ngslib I get the following error:
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
File "", line 1, in
File "/home/ryan/oncatator_virt_env/lib/python2.7/site-packages/ngslib/init.py", line 55, in
import wWigIO
ImportError: /home/ryan/oncatator_virt_env/lib/python2.7/site-packages/wWigIO.so: undefined symbol: cmpTwoBits32
I am not sure how to proceed now. Any ideas on how to resolve this?
PS: in a different environment I can install ngslib-1.1.20 and pysam-0.12.0.1. I can then import the ngslib module without any issue.
Thanks!
@Ryan, I'll ask someone to look into this.
I can't download the annotation file "oncotator_v1_ds_April052016.tar.gz". It needs username and password.
Thanks!
@Ryan Are you using BigWig datasources? (There are none in the default datasources). If you are not using any, just ignore the ngslib message. Oncotator should still work fine for you.
@LeeTL1220 I am not using any BigWig datasources like you mentioned so I will move forward and ignore the message. I did a little more testing today and ngslib 1.1.18 and 1.1.19 both give the same error. ngslib 1.1.20 imports with no error. Thanks for the help!
I noticed the output from oncotator removes the chr prefix from the chromosome location when outputting to VCF. I understand that the default output is a MAF format that wouldn't have the chr prefix. Would it be possible to have an option in oncotator that would keep that CHROM field intact?
Thanks!
Hi, I have followed your instructions to successfully create the virtualenv for oncotator-1.9.8.0 on MacOSx. But I am having troubles on $ python setup.py install step. It doesn't look like a dependency error but I am not sure, I searched it a lot but still without solution. Error line is;
ld: library not found for -lcrypto
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: Setup script exited with error: command 'cc' failed with exit status 1
pip freeze returns
bcbio-gff==0.6.2
biopython==1.66
Cython==0.24
enum34==1.1.2
futures==3.2.0
more-itertools==2.2
natsort==4.0.4
ngslib==1.1.18
nose==1.3.7
numpy==1.11.0
Oncotator==1.9.8.0
pandas==0.18.0
parse==1.8.2
pysam==0.14.1
python-dateutil==2.7.2
python-memcached==1.57
pytz==2018.3
PyVCF==0.6.8
shove==0.6.6
six==1.11.0
SQLAlchemy==1.0.12
stuf==0.9.16
Seems to be a C compiler issue, solved after updating Xcode.
ld: library not found for -lcrypto
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: Setup script exited with error: command 'cc' failed with exit status 1
I am working on MacOs Mojave and successfully created the virtual environment for oncotator-1.9.9.0. I also updated Xcode (I have now Xcode 10.1).
pip freeze returns:
bcbio-gff==0.6.2
biopython==1.66
Cython==0.24
enum34==1.1.2
futures==3.2.0
more-itertools==2.2
natsort==4.0.4
ngslib==1.1.18
nose==1.3.7
numpy==1.11.0
Oncotator==1.9.9.0
pandas==0.18.0
parse==1.9.0
pysam==0.15.2
python-dateutil==2.7.5
python-memcached==1.57
pytz==2018.9
PyVCF==0.6.8
shove==0.6.6
six==1.12.0
SQLAlchemy==1.0.12
stuf==0.9.16
Any idea of what I could try?
Thanks!
Hello,
to get the oncotator_v1_ds_April052016.tar.gz
BTW, compressing it with xz would speed up the downloads.
Hope it helps,
DK
can not Download UniProt Exact Match + Clinical
it shows 404 page not found
https://personal.broadinstitute.org/lichtens/oncobeta/tx_exact_uniprot_matches.AKT1_CRLF2_FGFR1.txt