Due to a minor hiccup in the release process for version 3.3, the tool documentation page is currently malfunctioning. We're working on getting that fixed asap.

ReduceReads very Slow for MT Reads

jfarrelljfarrell Posts: 35Member
edited February 2013 in Ask the GATK team

ReduceReads is very slow for MT reads. After it gets by the MT, it runs much faster (See output below)

Any ideas why and what to do to speed it up?

John

INFO 23:32:37,536 HelpFormatter - --------------------------------------------------------------------------------- INFO 23:32:37,545 HelpFormatter - The Genome Analysis Toolkit (GATK) v2.2-16-g9f648cb, Compiled 2012/12/04 03:46:58 INFO 23:32:37,545 HelpFormatter - Copyright (c) 2010 The Broad Institute INFO 23:32:37,545 HelpFormatter - For support and documentation go to http://www.broadinstitute.org/gatk INFO 23:32:37,551 HelpFormatter - Program Args: -R /unprotected/projects/genetics_program/resources/gatk_bundle/hg19/ucsc.hg19.fasta -T ReduceReads -I bam/LP6005113-DNA_E01.recal.bam -o LP6005113- DNA_E01.reduced.bam INFO 23:32:37,551 HelpFormatter - Date/Time: 2013/01/28 23:32:37 INFO 23:32:37,551 HelpFormatter - --------------------------------------------------------------------------------- INFO 23:32:37,551 HelpFormatter - --------------------------------------------------------------------------------- INFO 23:32:37,605 GenomeAnalysisEngine - Strictness is SILENT INFO 23:32:37,984 GenomeAnalysisEngine - Downsampling Settings: No downsampling INFO 23:32:37,992 SAMDataSource$SAMReaders - Initializing SAMRecords in serial INFO 23:32:38,073 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 0.08 INFO 23:32:38,113 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING] INFO 23:32:38,114 ProgressMeter - Location processed.reads runtime per.1M.reads completed total.runtime remaining INFO 23:33:14,010 ProgressMeter - chrM:1992 4.00e+04 35.9 s 15.0 m 0.0% 93.5 w 93.5 w INFO 23:35:18,038 ProgressMeter - chrM:2879 6.00e+04 2.7 m 44.4 m 0.0% 288.2 w 288.2 w INFO 23:37:06,320 ProgressMeter - chrM:3259 7.00e+04 4.5 m 63.9 m 0.0% 427.0 w 427.0 w INFO 23:39:03,457 ProgressMeter - chrM:3662 8.00e+04 6.4 m 80.3 m 0.0% 546.0 w 546.0 w INFO 23:41:16,174 ProgressMeter - chrM:4087 9.00e+04 8.6 m 95.9 m 0.0% 657.7 w 657.7 w INFO 23:43:46,243 ProgressMeter - chrM:4550 1.00e+05 11.1 m 111.3 m 0.0% 761.8 w 761.8 w INFO 23:46:47,501 ProgressMeter - chrM:4973 1.10e+05 14.2 m 2.1 h 0.0% 886.1 w 886.1 w INFO 23:49:57,085 ProgressMeter - chrM:5379 1.20e+05 17.3 m 2.4 h 0.0% 1002.1 w 1002.1 w INFO 23:52:52,173 ProgressMeter - chrM:5823 1.30e+05 20.2 m 2.6 h 0.0% 1081.7 w 1081.7 w INFO 23:54:28,697 ProgressMeter - chrM:7492 1.70e+05 21.8 m 2.1 h 0.0% 907.5 w 907.5 w INFO 23:55:45,484 ProgressMeter - chrM:7883 1.80e+05 23.1 m 2.1 h 0.0% 913.0 w 913.0 w INFO 23:57:16,597 ProgressMeter - chrM:8305 1.90e+05 24.6 m 2.2 h 0.0% 923.5 w 923.5 w INFO 23:59:07,109 ProgressMeter - chrM:8731 2.00e+05 26.5 m 2.2 h 0.0% 944.1 w 944.1 w INFO 00:01:16,623 ProgressMeter - chrM:9124 2.10e+05 28.6 m 2.3 h 0.0% 977.1 w 977.1 w INFO 00:04:12,150 ProgressMeter - chrM:9526 2.20e+05 31.6 m 2.4 h 0.0% 1031.4 w 1031.4 w INFO 00:06:51,054 ProgressMeter - chrM:9896 2.30e+05 34.2 m 2.5 h 0.0% 1076.2 w 1076.2 w INFO 00:09:31,477 ProgressMeter - chrM:10244 2.40e+05 36.9 m 2.6 h 0.0% 1120.9 w 1120.9 w INFO 00:12:57,847 ProgressMeter - chrM:10626 2.50e+05 40.3 m 2.7 h 0.0% 1181.3 w 1181.3 w INFO 00:16:48,872 ProgressMeter - chrM:11139 2.60e+05 44.2 m 2.8 h 0.0% 1234.5 w 1234.5 w INFO 00:20:54,282 ProgressMeter - chrM:11634 2.70e+05 48.3 m 3.0 h 0.0% 1291.4 w 1291.4 w INFO 00:25:23,381 ProgressMeter - chrM:12098 2.80e+05 52.8 m 3.1 h 0.0% 1357.2 w 1357.2 w INFO 00:30:01,695 ProgressMeter - chrM:12464 2.90e+05 57.4 m 3.3 h 0.0% 1433.2 w 1433.2 w INFO 00:34:41,008 ProgressMeter - chrM:12805 3.00e+05 62.0 m 3.4 h 0.0% 1508.2 w 1508.2 w INFO 00:39:41,462 ProgressMeter - chrM:13307 3.10e+05 67.1 m 3.6 h 0.0% 1568.4 w 1568.4 w INFO 00:45:21,827 ProgressMeter - chrM:13764 3.20e+05 72.7 m 3.8 h 0.0% 1644.6 w 1644.6 w INFO 00:51:15,645 ProgressMeter - chrM:14173 3.30e+05 78.6 m 4.0 h 0.0% 1726.7 w 1726.7 w INFO 00:57:38,039 ProgressMeter - chrM:14639 3.40e+05 85.0 m 4.2 h 0.0% 1807.2 w 1807.2 w INFO 01:06:06,413 ProgressMeter - chrM:15067 3.50e+05 93.5 m 4.5 h 0.0% 1930.9 w 1930.9 w INFO 01:15:07,742 ProgressMeter - chrM:15463 3.60e+05 102.5 m 4.7 h 0.0% 2063.0 w 2063.0 w INFO 01:23:17,067 ProgressMeter - chrM:15827 3.70e+05 110.6 m 5.0 h 0.0% 2176.0 w 2176.0 w INFO 01:31:08,225 ProgressMeter - chrM:16237 3.80e+05 118.5 m 5.2 h 0.0% 2271.6 w 2271.5 w INFO 01:32:08,631 ProgressMeter - chr1:3000534 1.17e+06 119.5 m 102.5 m 0.1% 12.3 w 12.3 w INFO 01:33:09,058 ProgressMeter - chr1:5169965 1.91e+06 2.0 h 63.2 m 0.2% 7.2 w 7.2 w INFO 01:34:09,530 ProgressMeter - chr1:7090404 2.65e+06 2.0 h 45.9 m 0.2% 5.3 w 5.3 w INFO 01:35:10,334 ProgressMeter - chr1:8806475 3.32e+06 2.0 h 37.0 m 0.3% 4.3 w 4.3 w INFO 01:36:10,654 ProgressMeter - chr1:10887467 4.08e+06 2.1 h 30.3 m 0.3% 3.5 w 3.5 w INFO 01:37:10,892 ProgressMeter - chr1:12756332 4.77e+06 2.1 h 26.1 m 0.4% 3.0 w 3.0 w INFO 01:38:11,087 ProgressMeter - chr1:14746000 5.29e+06 2.1 h 23.8 m 0.5% 18.5 d 18.4 d INFO 01:39:11,327 ProgressMeter - chr1:16699493 6.02e+06 2.1 h 21.0 m 0.5% 16.5 d 16.4 d INFO 01:40:11,606 ProgressMeter - chr1:18706430 6.86e+06 2.1 h 18.6 m 0.6% 14.8 d 14.8 d

Post edited by Geraldine_VdAuwera on
Tagged:

Best Answers

Answers

  • jfarrelljfarrell Posts: 35Member

    Yes, It is much faster with the latest version. Below is output with reducing 14 subjects with 30-50x coverage (previous example had just 1 bam file). It still gives some very long initial estimates for chrM but now finishes up in a few minutes. Great speed up!

    FYI--for some reason the XHMM was the only drop down choice on the submit form (both chrome and IE). I still see this behaviour on this form (http://gatkforums.broadinstitute.org/post/question) which I used for submitting the original question.

    INFO 12:34:46,005 SAMDataSource$SAMReaders - Done initializing BAM readers: total time 3.95 INFO 12:34:46,407 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING] INFO 12:34:46,407 ProgressMeter - Location processed.reads runtime per.1M.reads completed total.runtime remaining INFO 12:34:46,531 ReadShardBalancer$1 - Loading BAM index data for next contig INFO 12:34:46,537 ReadShardBalancer$1 - Done loading BAM index data for next contig INFO 12:35:16,413 ProgressMeter - chrM:1574 4.20e+05 30.0 s 71.4 s 0.0% 98.9 w 98.9 w INFO 12:36:16,417 ProgressMeter - chrM:5044 1.40e+06 90.0 s 64.2 s 0.0% 92.6 w 92.6 w INFO 12:37:16,420 ProgressMeter - chrM:7517 2.10e+06 2.5 m 71.4 s 0.0% 103.5 w 103.5 w INFO 12:38:16,560 ProgressMeter - chrM:9360 2.66e+06 3.5 m 78.9 s 0.0% 116.5 w 116.5 w INFO 12:39:16,563 ProgressMeter - chrM:12645 3.64e+06 4.5 m 74.1 s 0.0% 110.8 w 110.8 w INFO 12:40:21,334 ProgressMeter - chrM:16540 4.77e+06 5.6 m 70.2 s 0.0% 105.0 w 105.0 w INFO 12:41:21,400 ProgressMeter - chr1:783716 5.61e+06 6.6 m 70.4 s 0.0% 17.9 d 17.9 d INFO 12:42:28,493 ProgressMeter - chr1:999788 6.73e+06 7.7 m 68.7 s 0.0% 16.5 d 16.5 d INFO 12:43:33,989 ProgressMeter - chr1:1261743 7.85e+06 8.8 m 67.2 s 0.0% 15.0 d 15.0 d INFO 12:44:37,800 ProgressMeter - chr1:1522378 8.97e+06 9.9 m 65.9 s 0.0% 14.0 d 13.9 d

Sign In or Register to comment.