Update: July 26, 2019
This section of the forum is now closed; we are working on a new support model for WDL that we will share here shortly. For Cromwell-specific issues, see the Cromwell docs and post questions on Github.

how to run cromwell on large number of samples

blueskypyblueskypy Member ✭✭

I constructed a WES variant calling WDL pipeline to run on one sample. And I use the following command to run the pipeline in a SGE cluster

for i in ...;do
     nohup "java -Dconfig.file=sge.conf -jar cromwell-36.jar run fq2gvcf.wdl -i sample.$i.json" >$i.out 2>$i.err &

But when the number of sample is large, cromwell gives the error, which is probably due to the overhead of running too many instances of cromwell.

java.lang.OutOfMemoryError: unable to create new native thread

Can someone please tell how to solve this problem? one suggests to use the server mode instead of the run mode, but I'm not sure how to use server mode on a cluster I can only ssh into, and how to use server mode to run large number of samples?


Sign In or Register to comment.