Long idle times between tasks in GCP and tricks to reduce VM-related time
I am running a wdl script (WES somatic variant discovery) in GCP, launching it via wdl-runner (https://github.com/broadinstitute/wdl-runner), and I am observing that the time between tasks can reach 10 minutes or even more. During this time the Cromwell machine seems to be idle since nothing is apparently happening in the "Compute Engine" console.
By time between tasks I mean the time after one task has finished successfully and the VM machine is killed and the time that a new VM machine is created to execute the following task. I am experiencing this especially before and after the function Mutect2, which I am running in NIO mode (input files declared as
String at the task section for inputs so that files are read "in streaming").
Is there a way to speed up the transition between tasks? This idle time adds up to the time that takes to create and kill the VM machines (usually around 10-15 minutes overall), contributing a substantial extra time to the strict execution time of my wdl script: probably more than 20% of the running time is due to this VM-related operations and seemingly idle times.
Thanks in advance.