What is Docker?
Docker containers wrap software in a filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server, thus enabling portability of tools across operating systems.
FireCloud uses Docker to distribute tools and applications for use in its Methods. Docker allows applications and their dependencies to be packaged into discrete runtime environments, called containers.
Using Docker images, you can quickly deploy and run applications in Methods (i.e., Python or Matlab) without having to physically upload them. You can also use Docker images to import your own tools and applications for use in the Method Repository.
Docker containers are based on read-only environments, called images. Any change that is made to an image is carried out by adding a 'layer' on top of it. The dependencies of Docker layers define an ordering, and any layer that does not depend on any other layer is called a base image. Typically this layer will describe the operating system for the environment.
Docker containers are built from images, but feature a read-write layer on top, so, unlike images, are interactive and can store state. This means that you can run applications inside them. After any executions in your Docker container, you can save a snapshot of the resulting state as another image.
Docker Image Registry
Docker hosts a number of images that you can use to build your containers. These are stored in an online registry called Docker Hub, where they are organized into repositories. Docker Hub includes repositories that are are administrated by Docker, and contain images in which commonly-used software has been installed, including operating systems. These are typically used as base images. Docker Hub also hosts user repositories, containing images created and uploaded by Docker users. You are free to use images from any Docker Hub repository in building the image you reference in Methods (WDL).
You can also upload Docker images to the Broad Institute’s Docker Hub registry, where they are referenced by fixed image IDs. Once uploaded, you can use reference these images in FireCloud Methods. Workflows will execute the tools in series, inside the containers that they are installed in.