The job was aborted from outside Cromwell -- sporadic failures when running Cromwell locally

Hi -- I'm developing WDL workflows locally, and am working on a scatter-gather workflow specifically. The scattering works fine without errors (in this case, the Mutect1_Task), but then my jobs fail sporadically for the Gather section. There doesn't seem to be rhyme or reason to when they fail, and there is no informative error message. I've seen the same code fail sometimes and succeed other times. Below is the error message. Any ideas why this might happen?

[2018-02-26 10:45:29,62] [info] BackgroundConfigAsyncJobExecutionActor [aebb1bc5Mutect1_for_WES_PoN_Workflow.Mutect1_Task:0:1]: job id: 23625
[2018-02-26 10:45:29,62] [info] BackgroundConfigAsyncJobExecutionActor [aebb1bc5Mutect1_for_WES_PoN_Workflow.Mutect1_Task:1:1]: job id: 23624
[2018-02-26 10:45:29,62] [info] BackgroundConfigAsyncJobExecutionActor [aebb1bc5Mutect1_for_WES_PoN_Workflow.Mutect1_Task:0:1]: Status change from - to WaitingForReturnCodeFile
[2018-02-26 10:45:29,62] [info] BackgroundConfigAsyncJobExecutionActor [aebb1bc5Mutect1_for_WES_PoN_Workflow.Mutect1_Task:1:1]: Status change from - to WaitingForReturnCodeFile
[2018-02-26 10:46:10,58] [info] BackgroundConfigAsyncJobExecutionActor [aebb1bc5Mutect1_for_WES_PoN_Workflow.Mutect1_Task:0:1]: Status change from WaitingForReturnCodeFile to Done
[2018-02-26 10:46:17,74] [info] BackgroundConfigAsyncJobExecutionActor [aebb1bc5Mutect1_for_WES_PoN_Workflow.Mutect1_Task:1:1]: Status change from WaitingForReturnCodeFile to Done
_**
[2018-02-26 10:46:18,55] [error] WorkflowManagerActor Workflow aebb1bc5-0b99-4704-a5ff-bdc6e25a9a05 failed (during ExecutingWorkflowState): The job was aborted from outside Cromwell**_
java.lang.Exception: The job was aborted from outside Cromwell
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor$$anonfun$5.applyOrElse(WorkflowExecutionActor.scala:197)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor$$anonfun$5.applyOrElse(WorkflowExecutionActor.scala:146)
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:168)
at akka.actor.FSM.processEvent(FSM.scala:668)
at akka.actor.FSM.processEvent$(FSM.scala:662)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor.akka$actor$LoggingFSM$$super$processEvent(WorkflowExecutionActor.scala:41)
at akka.actor.LoggingFSM.processEvent(FSM.scala:801)
at akka.actor.LoggingFSM.processEvent$(FSM.scala:783)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor.processEvent(WorkflowExecutionActor.scala:41)
at akka.actor.FSM.akka$actor$FSM$$processMsg(FSM.scala:659)
at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:653)
at akka.actor.Actor.aroundReceive(Actor.scala:514)
at akka.actor.Actor.aroundReceive$(Actor.scala:512)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor.akka$actor$Timers$$super$aroundReceive(WorkflowExecutionActor.scala:41)
at akka.actor.Timers.aroundReceive(Timers.scala:44)
at akka.actor.Timers.aroundReceive$(Timers.scala:36)
at cromwell.engine.workflow.lifecycle.execution.WorkflowExecutionActor.aroundReceive(WorkflowExecutionActor.scala:41)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
at akka.actor.ActorCell.invoke(ActorCell.scala:496)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Best Answer

Answers

Sign In or Register to comment.