Cromwell 26: Retrying for you
Cromwell 25 was released a few weeks ago but already Cromwell 26 is available! In this release we improved Cromwell's ability to cope with failure, including adding cases where Cromwell will retry jobs that failed due to transient errors. Note that migration from version 25 to 26 is expected to take extra time as explained further below.
But before you do, have a look at the version highlights below!
Breaking Change: Failure metadata
Failure metadata will now be in a consistent JSON format, previously it varied depending on the originating Cromwell version. Failures will be an array of JSON objects, each representing a failure. See the Changelog for an example.
If at first you don't succeed...(re)try, (re)try again
Cromwell will retry your workflow when faced with transient errors from the Pipelines API (formerly known as JES). For example, when authentication fails (like this user faced) or it cannot access files, Cromwell will try a few more times to get past these errors.
New Configuration option
You can configure the number of I/O queries that Cromwell makes in the config file. This is mostly useful as a performance tuning option for the Pipelines API backend.
To promote reusing the same code, we packaged up our Docker hashes for the FireCloud team so they can fully enable Call Caching (aka Job Avoidance). Go team!
But it won't because there is new WDL syntax that supports
then. For example,
Boolean morning = ... String greeting = "good " + if morning then "morning" else "afternoon"
Migration takes time
It will take some additional time to upgrade to Cromwell 26 from Cromwell 25 (or a previous version of Cromwell). We can give you a rough estimate of how long (if you're curious), see the note in the Changelog.
Hello Scala 2.12
We upgraded the Lenthall and WDL4S repos, though we are still publishing 2.11 artifacts for each.
- Cromwell now does not slow down with large scatters.
- There was an issue with restarts, but it's all fixed now.
- Cromwell stopped hanging around when it's missing the WDL for a sub-workflow.
See the Changelog for more details.