This section of the forum is no longer actively monitored. We are working on a support migration plan that we will share here shortly. Apologies for this inconvenience.
How to realize and control scatter-gather function with Cromwell in a VM machine
Dear GATK team and other users,
I was trying to optimise my parameter in my 1 VM machine (24 cores) when running the data pre-processing workflow. Basically, I used the wdl script offered by gatk github page. my input contains multiple ubam files. My wdl script use scatter-gather to parallely processing these ubam files.
I feel so confused. I read paper, it said scatter-gather realize cluster level parallel computing which means I need multiple nodes and each nodes will independently process its work at the same time.
Now I only have 1 VM which contains 24 cores. does it means I only can realize multi-threads level parallel computing or actually I also can keep the scatter-gather function in my script and get benefit from it? and How?