Update: July 26, 2019
This section of the forum is now closed; we are working on a new support model for WDL that we will share here shortly. For Cromwell-specific issues, see the Cromwell docs and post questions on Github.

glob function is inconsistent between jes backend and local

I noticed I was getting a different output when running a task using glob on FireCloud versus testing locally with Cromwell so I wrote this little example. Neither of the output files contain the file paths I would expect. I assumed that glob1.txt would contain a path to three.unique.txt and four.unique.txt, whereas glob2.txt would contain paths to one.unique.txt and two.unique.txt. However, glob1.txt is blank and glob2.txt has 8 paths.

I'm not sure which back end is being used when run locally, but in searching github (https://github.com/broadinstitute/cromwell/search?utf8=✓&q=glob), it looks like one of the back ends is doing a recursive search which would explain this behavior:
toPath(path).glob(s"**/$pattern")

workflow globTestWorkflow {

    call makeFilesTask {}

    call checkGlobTask {
        input:
        file_glob1=makeFilesTask.glob1,
        file_glob2=makeFilesTask.glob2
    }
}

task makeFilesTask {

    command <<<
        touch one.unique.txt
        touch two.unique.txt

        mkdir Reads
        touch ./Reads/three.unique.txt
        touch ./Reads/four.unique.txt
    >>>

    output {
        Array[File] glob1 = glob("./Reads/*.unique.txt")
        Array[File] glob2 = glob("*.unique.txt")
    }

    runtime {
        docker: "mburger/crispr-seq"
    }

}

task checkGlobTask {
    Array[File] file_glob1
    Array[File] file_glob2

    command <<<
        echo ${sep=' ' file_glob1} > glob1.txt
        echo ${sep=' ' file_glob2} > glob2.txt
    >>>

    output {
        File out_txt1 = "glob1.txt"
        File out_txt2 = "glob2.txt"
    }

    runtime {
        docker: "mburger/crispr-seq"
    }
}

Best Answer

Answers

Sign In or Register to comment.