jweinstk · ✭
- Last Active
* Yes, call caching is enabled. * Yes, all input files live in GCS * Yes, please see github gist in my original post, which has the WDL. I was able to get Cromwell to submit jobs (much) faster after removing the call to the "size" command…
@kshakir , thanks for the reply! I am already reading in the file names using read_lines on a text file with all of the file paths. My scatter tasks don't need every single file available to every docker VM - the process is more to run a few tasks o…
@ChrisL Great - thank you (and the Cromwell team) so much for looking into it!! Looking forward to using the new 28 release.
@ChrisL , I parsed out the Success array from the log, and it has 493 elements. I've examined a couple of the shards that are missing, and I don't quite see why they are not included here. For example, shard-14 is missing from the Success array, but…
@Ruchi - I can confirm that arrays that I am indexing should be the same length.
Sure, the WDL script I'm using is very similar to this . The array that I'm scattering over is an array of file names, which are read in using read_lines (you may notice this does not happen in the linked wdl file - please ignore that discrepancy. …