Holiday Notice:
The Frontline Support team will be slow to respond December 17-18 due to an institute-wide retreat and offline December 22- January 1, while the institute is closed. Thank you for your patience during these next few weeks. Happy Holidays!

Pre-start hook?

I'm trying to install a monitoring agent (from Dynatrace / DataDog) on the VMs that get launched from our workflow and discovered the agents can't be directly installed in Docker containers (we're running Google Pipeline API). Both products say that the way to run these monitoring agents in a Dockerized environment is to perform a docker run of a particular image they have that contains their monitoring software. Seeing as how Cromwell issues it's own docker run of the image we've created is there some kind of pre-start hook that we can tap in to so that we can run the monitoring container before our workflow container is run?

Failing that, does anyone have an idea where in the Cromwell source code the docker run command is issued?

Tagged:

Answers

  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev admin

    I answered this via the Google Genomics Discuss mailing group, but I wanted to make sure the answer is available here too:

    This might not be exactly what you're after but did you investigate the monitoring script option in Cromwell's PAPI backend? http://cromwell.readthedocs.io/en/develop/wf_options/Google/#google-pipelines-api-workflow-options

    In the Google cloud, Cromwell never issues the docker run itself, it submits requests to PAPI (pipelines API) containing the docker image, a command to run, and the set of inputs and outputs to localize/delocalize from the VM the command ends up running on.

  • Hi Chris, yes, I saw the post over at the Google group and I'm still trying at the moment to get the Datadog container running before the container that contains our pipeline stack (using the monitoring script option). Stay tuned.

Sign In or Register to comment.