Pre-start hook?

I'm trying to install a monitoring agent (from Dynatrace / DataDog) on the VMs that get launched from our workflow and discovered the agents can't be directly installed in Docker containers (we're running Google Pipeline API). Both products say that the way to run these monitoring agents in a Dockerized environment is to perform a docker run of a particular image they have that contains their monitoring software. Seeing as how Cromwell issues it's own docker run of the image we've created is there some kind of pre-start hook that we can tap in to so that we can run the monitoring container before our workflow container is run?

Failing that, does anyone have an idea where in the Cromwell source code the docker run command is issued?

Tagged:

Answers

  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev

    I answered this via the Google Genomics Discuss mailing group, but I wanted to make sure the answer is available here too:

    This might not be exactly what you're after but did you investigate the monitoring script option in Cromwell's PAPI backend? http://cromwell.readthedocs.io/en/develop/wf_options/Google/#google-pipelines-api-workflow-options

    In the Google cloud, Cromwell never issues the docker run itself, it submits requests to PAPI (pipelines API) containing the docker image, a command to run, and the set of inputs and outputs to localize/delocalize from the VM the command ends up running on.

  • Hi Chris, yes, I saw the post over at the Google group and I'm still trying at the moment to get the Datadog container running before the container that contains our pipeline stack (using the monitoring script option). Stay tuned.

Sign In or Register to comment.