How to develop WDL workflows using Docker images locally which can be used seamlessly in the cloud

dannykwellsdannykwells San FranciscoMember ✭✭

Hi Broad/WDL folks,

I love WDL and we're really excited it's potential (good work!)
One question that I have about it is about what the workflow looks like for developing WDL programs locally (on a Mac) and then deploying those on the cloud (Google). How do you do this at Broad?

From here (https://docs.google.com/document/d/1jv4FFWC0RJUWoQagh7wA7L7K6lHNz72zGttwEfVKgnA/edit?ts=5601ad23#heading=h.16u3o9piptvm) it seems that you work within a VM when making Docker images on a Mac (although this is no longer required in the latest Docker release - https://docs.docker.com/engine/installation/mac/ - is this required if I want to call containers inside a WDL program? Said another way, is it possible to use containers made on a Mac within a WDL program from the standard Mac terminal? In turn, are there substantial changes to these programs needs before they can be deployed on a cluster?

In general, any help using WDL to write pipelines using Docker containers holding needed functions (bwa, picard, GATK) that can be developed and executed locally on a mac and ported easily to a cluster computing environment would be really helpful!

Thank you!

Best Answer

Answers

  • dannykwellsdannykwells San FranciscoMember ✭✭

    Hi @KateN, thank you! This helped and I have WDL et al running both on my laptop and GCP!

Sign In or Register to comment.