We've moved!
For WDL questions, see the WDL specification and WDL docs.
For Cromwell questions, see the Cromwell docs and please post any issues on Github.

glob function is inconsistent between jes backend and local

I noticed I was getting a different output when running a task using glob on FireCloud versus testing locally with Cromwell so I wrote this little example. Neither of the output files contain the file paths I would expect. I assumed that glob1.txt would contain a path to three.unique.txt and four.unique.txt, whereas glob2.txt would contain paths to one.unique.txt and two.unique.txt. However, glob1.txt is blank and glob2.txt has 8 paths.

I'm not sure which back end is being used when run locally, but in searching github (https://github.com/broadinstitute/cromwell/search?utf8=✓&q=glob), it looks like one of the back ends is doing a recursive search which would explain this behavior:
toPath(path).glob(s"**/$pattern")

workflow globTestWorkflow {

    call makeFilesTask {}

    call checkGlobTask {
        input:
        file_glob1=makeFilesTask.glob1,
        file_glob2=makeFilesTask.glob2
    }
}

task makeFilesTask {

    command <<<
        touch one.unique.txt
        touch two.unique.txt

        mkdir Reads
        touch ./Reads/three.unique.txt
        touch ./Reads/four.unique.txt
    >>>

    output {
        Array[File] glob1 = glob("./Reads/*.unique.txt")
        Array[File] glob2 = glob("*.unique.txt")
    }

    runtime {
        docker: "mburger/crispr-seq"
    }

}

task checkGlobTask {
    Array[File] file_glob1
    Array[File] file_glob2

    command <<<
        echo ${sep=' ' file_glob1} > glob1.txt
        echo ${sep=' ' file_glob2} > glob2.txt
    >>>

    output {
        File out_txt1 = "glob1.txt"
        File out_txt2 = "glob2.txt"
    }

    runtime {
        docker: "mburger/crispr-seq"
    }
}

Best Answer

Answers

Sign In or Register to comment.