For WDL questions, see the WDL specification and WDL docs.
For Cromwell questions, see the Cromwell docs and please post any issues on Github.
You can't spell Cromwell without CWL
By Jeff Gentry, Software Engineering Manager, Data Sciences Platform
In January I reported that Cromwell was expanding workflow language support beyond WDL via a concept we call the Workflow Object Model (WOM). In that post I announced that development towards supporting the Common Workflow Language (CWL) was already underway. As of today, that work is complete and we have released Cromwell version 32 which supports running CWL workflows.
The CWL community has grown into a thriving ecosystem over the last three years. Many groups have been developing CWL workflows and there’s a growing suite of services with first class support, such as, repositories like Dockstore, editors like Rabix Composer, and engines like Cromwell. Our goal with Cromwell has always been to provide the same seamless scalability to as many users as possible. With the bioinformatics community coalescing on a combination of both CWL and WDL workflows, we feel it was important that users shouldn’t be restricted by a language choice.
“Having an additional runner with first class CWL support demonstrates the portability and value of interoperable specifications coming out of the Open Bioinformatics and GA4GH communities.” - Brad Chapman, Senior Research Scientist in the Bioinformatics Core at Harvard Chan School
After many years of an increasingly fractured landscape, the bioinformatics world is now moving towards defining standards that promote interoperability of systems and tools.
This will allow people to redirect energy away from reinventing the wheel and towards making a scientific impact. With efforts such as the Global Alliance For Genomics & Health (GA4GH) and the Open Bioinformatics Foundation as well as large projects such as the Data Biosphere focusing on WDL and CWL, Cromwell can provide them with a single high quality, scalable system which can handle all of their workflow needs.
“Cromwell’s capacity to process CWL provides Australian Genomics with a freely available, highly scalable solution for our workflows. It prepares clinical Flagship projects to process genomic data from large cohorts on both commercial cloud and local HPC environments. It also gives project owners full control over where their data is stored — a key requirement to align with health privacy legislation and patient consent. The availability of our workflows in CWL Cromwell also allows us to harmonise results from different projects regardless of where data have been generated, empowering both clinical understanding and research linkages." - A/Prof Oliver Hofmann, Australian Genomics and co-lead of the GA4GH Large Scale Genomics Work Stream with Thomas Keane
We have been working with a number of groups to beta test Cromwell’s CWL support by running their workflows, including the Center for Data Intensive Science (CDIS) at the University of Chicago who develop the Genomic Data Commons workflows; the developers of the cancer workflows at the McDonnell Genome Institute at Washington University; as well as running bcbio-nextgen at the University of Melbourne's Centre for Cancer Research, AstraZeneca's Scientific Computing Platform and
Harvard Medical School's Orchestra2 cluster. We’re proud to be hearing numerous success stories, but don’t take my word for it ...
“Thanks to the new CWL support, we're running bcbio validations and analysis workflows using Cromwell. Cromwell’s wide platform support, including clusters with shared filesystems, allows us to continue to transition all our analyses to use CWL since we can now support users in traditional HPC environments” - Brad Chapman.
“We’ve been able to run our latest alignment workflow on a SLURM cluster using Cromwell’s new CWL support. In the past we’ve been limited in our ability to take advantage of parallelism in our workflows, with Cromwell we can now achieve that goal.” - Jeremiah Savage, Bioinformatician at CDIS
We’re looking forward to hearing about your experiences as well.