We've moved!
This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!

MarkDuplicatesSpark not respecting --conf 'spark.executor.cores=4' option

riederdriederd Innsbruck Medical UniversityMember

Hi,

I'm trying to run gatk MarkDuplicatesSpark (v 4.1.4.1) locally, so not on a spark cluster, and provided the option --conf 'spark.executor.cores=4' to tell MarkDuplicatesSpark to use only 4 cores on the machine. However when I check the system load with e.g. top I see that all 44 cores of the system are used by MarkDuplicatesSpark. What am I doing wrong?

command:
gatk MarkDuplicatesSpark \
--tmp-dir /local/scratch/tmp \
-I Control_aligned.bam \
-O Control_aligned_sort_mkdp.bam \
-M Control_aligned_sort_mkdp.txt \
--create-output-bam-index true \
--read-validation-stringency LENIENT \
--conf 'spark.executor.cores=4'

Best
Dietmar

Issue · Github
by bhanuGandham

Issue Number
6324
State
open
Last Updated
Assignee
Array
Milestone
Array

Answers

Sign In or Register to comment.