We've moved!
For WDL questions, see the WDL specification and WDL docs.
For Cromwell questions, see the Cromwell docs and please post any issues on Github.

Is there a way to tell Cromwell to ignore the "runtime section" of the tasks in a WDL script?

Is there a way to tell Cromwell to ignore the "runtime section" of the tasks in a WDL script? Our Linux servers don't let us use Docker, so it would be nice to be able to write a single WDL script that will run in three different situations: (1) Firecloud, (2) on my local Mac using a Docker image, and (3) on one of our Linux servers.

At the moment, I can run the same WDL script in all three situations, but for #3, at the moment I have to manually edit the WDL script to comment out the runtime sections. Hopefully, there's a better way, but at the moment, this is the only way I know how to accomplish this goal. (Short of writing a little program to strip out the runtime sections programmatically.)

I've tried defining a Boolean input parameter for the script and then doing something along the lines of

runtime {
    docker: if useDocker then "nessus/mite-seq-process" else ""

But unfortunately, that doesn't work. If useDocker is false, Cromwell will try to use a Docker image with the empty string as its name. That fails, of course.


Best Answer


  • RuchiRuchi Member, Broadie, Moderator, Dev admin

    Hey @nessus42,

    There is no way to selectively ignore certain runtime attributes, but you can configure your own Local backend that ignores docker by removing the submit-docker stanza in the default Local backend.
    A sample workflow options file:

    Local {
          actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
          config {
            # Limits the number of concurrent jobs
            #concurrent-job-limit = 5
            run-in-background = true
            # `script-epilogue` configures a shell command to run after the execution of every command block.
            # If this value is not set explicitly, the default value is `sync`, equivalent to:
            # script-epilogue = "sync"
            # To turn off the default `sync` behavior set this value to an empty string:
            # script-epilogue = ""
            runtime-attributes = """
            String? docker
            String? docker_user
            submit = "/bin/bash ${script}"
            #submit-docker = """
            #docker run \
            #  --rm -i \
            #  ${"--user " + docker_user} \
            #  --entrypoint /bin/bash \
            #  -v ${cwd}:${docker_cwd} \
            #  ${docker} ${script}
            # Root directory where Cromwell writes job results.  This directory must be
            # visible and writeable by the Cromwell process as well as the jobs that Cromwell
            # launches.
            root = "cromwell-executions"
            filesystems {
              local {
                localization: [
                  "hard-link", "soft-link", "copy"
                caching {
                  # When copying a cached result, what type of file duplication should occur. Attempted in the order listed below:
                  duplication-strategy: [
                    "hard-link", "soft-link", "copy"
                  # Possible values: file, path
                  # "file" will compute an md5 hash of the file content.
                  # "path" will compute an md5 hash of the file path. This strategy will only be effective if the duplication-strategy (above) is set to "soft-link",
                  # in order to allow for the original file path to be hashed.
                  hashing-strategy: "file"
                  # When true, will check if a sibling file with the same name and the .md5 extension exists, and if it does, use the content of this file as a hash.
                  # If false or the md5 does not exist, will proceed with the above-defined hashing strategy.
                  check-sibling-md5: false
            default-runtime-attributes {
              failOnStderr: false
              continueOnReturnCode: 0
  • nessus42nessus42 Member

    Thanks Ruchi!

    Unfortunately, Cromwell still tries to use Docker even if I use the config file that you provided. (I know that it's actually using the config file b/c if I introduce a syntax error into it, then Cromwell dies quickly, complaining about the syntax error.)

    Even while using the provided Cromwell config file, I see this in the terminal output:

    executing: docker run \
      --rm -i \
      --entrypoint /bin/bash \
      -v /Volumes/Ext-SSD/dalan-ext/s.big/local/wdl/mite-seq/cromwell-executions/fetchAndProcessFiles/51def993-4991-42b6-9a7c-cb54f4f945e2/call-expandPreMadeDataDir:/cromwell-executions/fetchAndProcessFiles/51def993-4991-42b6-9a7c-cb54f4f945e2/call-expandPreMadeDataDir \
      nessus/[email protected]:6a3ced61ebfd7056f6177c941e7d42adbfcf724c53796f67b2d8f5568f9bc562 /cromwell-executions/fetchAndProcessFiles/51def993-4991-42b6-9a7c-cb54f4f945e2/call-expandPreMadeDataDir/execution/script

    Thanks for your help!
    |>oug ([email protected])

  • nessus42nessus42 Member

    That seemed to work. Thanks!!!

  • nessus42nessus42 Member

    P.S. How do I get this forum to say that I'm a Broadie? My email address is set to "[email protected]".

  • ChrisLChrisL Cambridge, MAMember, Broadie, Dev admin

    I'm going to ping the existing ticket that requested optional docker values with this thread.

Sign In or Register to comment.