Forum Login Issue:
Currently the "Log in with Google" button redirects you to a "Page not found." This is an issue that our forum vendors are working on fixing. In the meantime, while on the "Page not found" you can edit the URL to delete the second gatk, firecloud, or wdl (depending on what subforum you are acessing).
ex: https://gatkforums.broadinstitute.org/gatk/gatk/entry/...

Something about job management

y_zhang88y_zhang88 Member
edited December 2017 in Ask the WDL team

Hi,
I am a new user and I really want to know how to manager jobs by WDL. For example, task will be run if the CPU usage is below 90% and memory is over 2G, otherwise it will be suspended until the condition is met. Does WDL contain this function?

Answers

  • kshakirkshakir Broadie, Dev

    Cromwell doesn't support this level of functionality for Local backends, but it can work with a job scheduler such as Grid Engine / SLURM / TORQUE / etc. to support resource requirements.

    WDL allows one to specify runtime attributes per task. One can configure Cromwell to pass the runtime attributes to the job scheduler during job submission. Under the documentation for setting up cromwell with an HPC custer there is an example that uses Grid Engine. Depending on who administers your cluster, most clusters usually support requesting a minimum amount of memory be available before a job will run. In the example, this is done with a flag -l mem_free= and then reading the runtime attribute memory as Float? memory_gb. NOTE: the docs under "memory" explain why there's a difference between the two names.

    The builtin cpu attribute is meant to request 1 whole CPU. Your concept of percentages is new to me, but if your job scheduler supports cpu percentages, in theory in your cromwell configuration file you could create a custom runtime-attribute, say Int? cpu_pct and then wire that in to your custom submit the same way as memory.

Sign In or Register to comment.