GATK on Google Cloud

Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin
edited January 9 in Pipelining Options

At this time we are able to offer two services for running WDL workflows on Google Cloud using the Cromwell execution engine and the Google Pipelines API. Note that while access to both of these services is free, compute and storage are subject to Google Cloud pricing, charged directly by Google (Broad does not apply any markup).

Google Pipelines API + WDL_Runner

These open-source command-line utilities were developed by Google Cloud to enable execution of analysis pipelines on the Google Cloud platform. The Google Pipelines API is a pipeline execution service that runs jobs on the Google Compute Engine (GCE). WDL_Runner is a package that provides support for running workflows written in WDL through the Pipelines API, via a Cromwell server that is spun up and torn down ad-hoc by WDL_Runner itself. See the GATK on Google Genomics tutorial for a hands-on demonstration of how to run one of our Best Practices production workflows on this service.

FireCloud: Our secure, freely accessible cloud-based analysis portal

This web-based application is provided as a freely accessible service by the Broad Institute's Data Sciences Platform, where GATK itself is also developed. FireCloud provides both GUI (point-and-click) and API access to a persistent Cromwell execution server that manages submissions to the Google Pipelines API. In addition to the core pipeline execution service, the FireCloud platform also includes functionality for data management, a data library of published datasets (including TCGA data) and a method repository for managing and sharing workflows. The platform as a whole is designed to empower analysts, tool developers and production managers to perform large-scale analysis, engage in data curation, and store or publish results. You can find out more about how FireCloud can make it easier for you to run GATK Best Practices pipelines here.

Post edited by Geraldine_VdAuwera on
Sign In or Register to comment.