Service Alert: 10/10/18
Load issues have been resolved and FireCloud is now catching up. If you are still stuck in a queued state in an hour, please let us know.
Latest Release: 10/09/18
Release Notes can be found here.

Jobs say `Requesting Execution Token` since last night

mshandmshand Member, Broadie, Dev

Sometime yesterday afternoon all of my submissions started showing up as Requesting Execution Token for 6+ hours. This morning they haven't moved so I'm aborting them and restarting them. So far the restarted workflows just have tasks that are Queued in Cromwell. Wondering if this is a known issue, or I've got something wrong with my wdls?

Best Answer

  • KateNKateN Cambridge, MA admin
    Accepted Answer

    Beginning last night (5/17) around 10 pm EST, we noticed a large influx of jobs (though it isn't too far to say that a job submitted yesterday afternoon could have encountered this issue and we hadn't been alerted yet). Cromwell is currently running the maximum number of jobs we allow, which means there is a backlog waiting in the QueuedInCromwell state.

    I'm working to find out more information about how long this delay might be, but for now we ask for your patience.

Answers

  • KateNKateN Cambridge, MAMember, Broadie, Moderator admin
    Accepted Answer

    Beginning last night (5/17) around 10 pm EST, we noticed a large influx of jobs (though it isn't too far to say that a job submitted yesterday afternoon could have encountered this issue and we hadn't been alerted yet). Cromwell is currently running the maximum number of jobs we allow, which means there is a backlog waiting in the QueuedInCromwell state.

    I'm working to find out more information about how long this delay might be, but for now we ask for your patience.

  • KateNKateN Cambridge, MAMember, Broadie, Moderator admin

    To follow up on this thread, we were able to find a solution. Details can be found here, and if you have any further issues with it, please report them there.

Sign In or Register to comment.