Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office on November 11th and 13th 2019, due to the U.S. holiday(Veteran's day) and due to a team event(Nov 13th). We will return to monitoring the GATK forum on November 12th and 14th respectively. Thank you for your patience.

ways to speed up Indel Realinger

Dear GATK-Team,

I am trying to speed up the Indel Realigner step in my pipeline. I have tried Queue, using number of scatters (-nsc) equal to 8; I noticed the jobs that are "scattered" finish relatively quickly (within 2 hours), but the Gather step is taking the bulk of the processing time (over 6 hours). I suspect this slow-down is because the scheduler for the cluster automatically assigns the Gather job to be done on just one core.

As I don't know enough about Queue and .scala to increase the number of cores for the Gather job, I was thinking of other ways to get the same output, but save time. Reading this post by @neilw in the forum (http://gatkforums.broadinstitute.org/discussion/3051/parallelizing-printreads), I was wondering if this approach would work:

1) Use RealignmentTargetCreator to get list of target intervals for realignment
2) Run 100 parallel jobs of IndelRealigner, passing to each the same (whole) input .bam and list of target intervals from step 1, but a unique portion of the genome using the -L option (intervals)
3) Merge the 100 realigned .bams with samtools to get single re-aligned .bam

Thanks a lot for your advice,
Steve

Best Answer

Answers

  • SheilaSheila Broad InstituteMember, Broadie admin

    @newGATKuser
    Hi Steve,

    That looks fine, but you can run RealignerTargetCreator on the unique intervals instead of starting using the intervals at the IndelRealigner step.

    -Sheila

  • newGATKusernewGATKuser CaseMember

    Hi Sheila,

    Thank you for the reply! Sorry, I just realized I wasn't very clear in my original post. The intervals I would use in step 2 with the -L option are actually 100 equal parts spanning the whole genome (so IndelRealigner would create a re-aligned .bam for 1/100th of the genome). In that case, I probably won't get a speed-up passing these intervals to RealignerTargetCreator?

    Steve

  • newGATKusernewGATKuser CaseMember

    Hi Sheila,

    Thank you for the good point! I had not thought of how Indel Realigner would treat reads that span intervals. I will try splitting by contigs to avoid this problem.

    Sincerely,
    Steve

Sign In or Register to comment.