Holiday Notice:
The Frontline Support team will be slow to respond December 17-18 due to an institute-wide retreat and offline December 22- January 1, while the institute is closed. Thank you for your patience during these next few weeks. Happy Holidays!

Why does this work?

I have found a weird issue in one of my wdl scripts, where specifying a folder path to a File? variable localises the folder into the docker runtime, and allows you to access the folder in the command section of your task.

The stripped down version of the script is here, am I wrong to think that this should not be possible?

task virulenceFind {
    File? database

    command {
        date
        ls ${default="/" database}
    }

    runtime {
        docker: "ubuntu:16.04"
    }

    output {
        String log = read_string(stdout())
    }
}

workflow virulence {

    call virulenceFind

    output {
        File log = virulenceFind.log
    }
}

settings.json

{
  "virulence.virulenceFind.database": "/home/ubuntu/Documents/"
}

Make sure the path exists on your system, so change ubuntu to your local username.

Answers

  • ThibThib CambridgeMember, Broadie, Dev ✭✭

    Hi @Redmar_van_den_Berg, the deal with directories is more in the line of "there are definitely things that don't work" than "definitely nothing works". In some circumstances it's possibly you end up in a supported case. For example here you're on a shared file system backend which moves files differently than if it were on a cloud backend with a cloud filesystem. The same workflow on the Pipelines API backend very likely would not work.
    We are working right now to support Directories in a more consistent way and this should eventually be expected to work.

  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev admin

    There's discussion going on in the OpenWDL community about adding a Directory type but I suspect that it'll be its own type, and not just an overload of File.

    So, I think this is a "shouldn't work" bug, and proper WDL directory support should be coming at some point in the near future.

  • Redmar_van_den_BergRedmar_van_den_Berg Member ✭✭

    hi @Thib and @ChrisL , thanks for the answer. What is the best place to keep an eye on the development of WDL directory support? Right now I'm using a bash task that soft-links the directory structure into the execute folder as a workaround, but that's not a very reliable method to work with input (or output) folders.

  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev admin

    Oops - I forgot to paste the PR: https://github.com/openwdl/wdl/pull/173

    The ticket is a little bit stale right now and needs a rebase, but work on it should spring back into life pretty soon.

Sign In or Register to comment.