Heads up:
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.
Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.
Attention:
We will be out of the office for a Broad Institute event from Dec 10th to Dec 11th 2019. We will be back to monitor the GATK forum on Dec 12th 2019. In the meantime we encourage you to help out other community members with their queries.
Thank you for your patience!

BaseRecalibrator doesn't respect -Xmx and -Xms settings

GATK version: version 2.1-12-g2d7797a

I'm using open grid engine via "starcluster", and have cluster nodes with 21G of ram. I have h_vmem as a requestable resource.

My qsub command requests "-l h_vmem=7G" and the java command is "java -Xms1G -Xmx5G -jar GenomeAnalysisTKLite.jar -T BaseRecalibrator ..."

the linux kernel/open grid engine kills the job:
main|node003|W|job 2271 exceeds job hard limit "h_vmem" of queue "[email protected]" (11716157440.00000 > limit:7516192768.00000) - sending SIGKILL

I've been reading about what I might be not specifying when trying to constrain the process, and perhaps I need to also add -XX:MaxPermSize to something specific ?

Best Answer

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MA admin
    Accepted Answer

    Hi Mike, I'm going to guess that when it invokes the Rscript, the R run is not subject to the limitations placed on the java job. And I don't think we've ever had a reason to build in an option to disable plotting. If you want to do it yourself and make a patch, we'd be happy to look at it.

Answers

  • I've been testing this a lot today -- it happens almost unpredictably, but with 1 caveat: it seems to be invoking that R script when it decides to take up 2x the amount of available heap space allocated to it. I've also added the -XX:MaxPermSize option, but the process will still take up 2x the amount of heap space allowed.

    It doesn't seem like you can disable plotting, why is that?

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin
    Accepted Answer

    Hi Mike, I'm going to guess that when it invokes the Rscript, the R run is not subject to the limitations placed on the java job. And I don't think we've ever had a reason to build in an option to disable plotting. If you want to do it yourself and make a patch, we'd be happy to look at it.

  • mike_lyonsmike_lyons Member
    edited November 2012

    Actually -- you already have an option to disable the plotting in the BaseRecalibrator.
    in package org.broadinstitute.sting.gatk.walkers.bqsr

    public class RecalibrationArgumentCollection {
     //line 173 from most recent github update
      @Hidden
      @Argument(fullName = "no_plots", shortName = "np", required = false, doc = "does not generate any plots -- useful for queue    scatter/gathering")
    public boolean NO_PLOTS = false;
     // ...
    }
    
    public class BaseRecalibrator extends LocusWalker<Long, Long> implements TreeReducible<Long> {
    // line 278in most recent github update
    @Override
        public void onTraversalDone(Long result) {
            logger.info("Calculating quantized quality scores...");
            quantizeQualityScores();
    
            logger.info("Writing recalibration report...");
            generateReport();
            logger.info("...done!");
    
            if (!RAC.NO_PLOTS) {
                logger.info("Generating recalibration plots...");
                generatePlots();
            }
    
            logger.info("Processed: " + result + " sites");
        }
     //...
    }
    
  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Oh yeah, there you go -- I hadn't run into that one yet. Thanks for pointing it out!

Sign In or Register to comment.