idea for enhancment : autosize

esalinasesalinas BroadMember, Broadie ✭✭✭

WDL tasks localize during JES runs using WDL like so:

task myTask {
File myFileToLocalize
Array[File] otherFilesToLocalize

...
command <<<
#process ${myFileToLocalize} ....

#process ${otherFilesToLocalize sep="somesep"} ...

>>>
...


also, WDL tasks can specify disk size for attached disk to help ensure that there is sufficient disk-space for files like so (in the example a user-configurable setting):

task diskTask {

    Int diskGB

.....

runtime {
disks: "local-disk ${diskGB} HDD"

...

It would seem like a good idea to have a special variable to automatic allocation of disk size of localized files. For example instead of the call being like

call diskTask {
input:
...
    diskGB=100
}

or

call diskTask {
input:
...
    diskGB=prepTask.calcDisk
}

or

call diskTask {
input:
...
    diskGB=size(someInputFile)
}

that maybe the variable could be like :

call bigTask {
inputs:
fileOne=someFileOfUnknownSize,
fileTwo=anotherFileOfUnknownSize,
diskGB=autoSize()+autoSize()*0.2
}

where "autoSize" automatically calculates sizes for files localized and adds them and I did some "+*" to add some factor from that (to have space for output files from the run) (in this case 20% of the disk for the output)

I wonder about people's thoughts on this?

Issue · Github
by Geraldine_VdAuwera

Issue Number
1928
State
open
Last Updated
Assignee
Array
Milestone
Array

Answers

Sign In or Register to comment.