Does cromwell prioritise finishing workflows

Evan_Benn

Say I have a 2 task workflow

workflow A
   call B
   call C

If I start 10,000 A workflows quickly, will cromwell schedule 10,000 B calls before any C calls? (C calls start after B calls end, so I think this is the behaviour).

I think this is the behaviour and it is sensible and understandable. What techniques can I use to try to get C calls prioritised over B calls, so that the workflows start finishing sooner? Are there any built in cromwell ways to do this?

Ideas I had; have a different backend for B and C, with the C backend always prioritised over B. Or similarly have a backend parameter that does the same thing. This is a bit annoying as it starts to tie your 'portable' wdl file tightly to the backend implementation.

Thanks

dario_romagnoli

I think you should use scatter over an array of inputs.
Something like this

workflow A {
    Array[File] array_of_files
    scatter (file in array_of_files) {
        call B {input: file=file}
        call C {input: file=B.output}
    }
}

dario_romagnoli

I think you should use scatter over an array of inputs.
Something like this

workflow A {
    Array[File] array_of_files
    scatter (file in array_of_files) {
        call B {input: file=file}
        call C {input: file=B.output}
    }
}

Ruchi

Agreed with @dario_romagnoli -- for scatter jobs that's exactly what should happen, you should start seeing results from task C if some task B jobs complete faster.

dario_romagnoli

(deleted message: wrong discussion)