Notice:
If you happen to see a question you know the answer to, please do chime in and help your fellow community members. We encourage our fourm members to be more involved, jump in and help out your fellow researchers with their questions. GATK forum is a community forum and helping each other with using GATK tools and research is the cornerstone of our success as a genomics research community.We appreciate your help!

Test-drive the GATK tools and Best Practices pipelines on Terra


Check out this blog post to learn how you can get started with GATK and try out the pipelines in preconfigured workspaces (with a user-friendly interface!) without having to install anything.

How can i set cores that MarkDuplicatesSpark uses?

I use command --conf 'spark.executor.cores=5', but the MarkDuplicates still all available core on my service.
Besides, my service got 16 cores in total.
How can fixed it ?

Answers

  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin

    Hi @Zea1nfO

    Can you please post the exact MarkDuplicates command you are using and the version of GATK you are using. Also please post information on how you are determining the number of cores used by MarkDuplicates.

  • Zea1nfOZea1nfO Member

    Hi @bhanuGandham
    here is my command :
    gatk MarkDuplicatesSpark \
    -I input.bam \
    -O output.bam \
    -M metrics.txt \
    -VS SILENT \
    --conf 'spark.executor.cores=5' \
    --tmp-dir /data/tmp/ \
    --OBI false \
    --optical-duplicate-pixel-distance 2500

    Is there something wrong with it?
    thanks a lot

  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin

    Hi @Zea1nfO

    Can you please try to run this command and see if it works. I have replaced --conf 'spark.executor.cores=5' with --spark-master local[5]

    gatk MarkDuplicatesSpark \
    -I input.bam \
    -O output.bam \
    -M metrics.txt \
    -VS SILENT \
    --spark-master local[5] \
    --tmp-dir /data/tmp/ \
    --OBI false \
    --optical-duplicate-pixel-distance 2500

  • Zea1nfOZea1nfO Member

    hi @bhanuGandham
    i will try it
    thanks a lot :)

Sign In or Register to comment.