Error evaluating output files that serve as input files for following step

YatrosYatros Seattle, WA, USAMember

Hello,

I have 16 paired end fastq files from 8 different runs for a single sample and I'm trying to map all of them individually with bwa mem software and combining the output sam files afterwards with the picard MergeSamFiles. However, every time I try to run the wdl script, I get the error

[error] WorkflowManagerActor Workflow c7822e7c-6361-4aa9-b7f8-0483798baf72 failed (during ExecutingWorkflowState): Could not evaluate bwa1.out = "${sample_name}_${run}.sam}" java.lang.RuntimeException: Could not evaluate bwa1.out = "${sample_name}_${run}.sam}"

I'm using wdltool-0.14 and cromwell 29 locally in Linux CentOS 7 with java openjdk version "1.8.0_141".

The simplified wdl code for 2 of the 8 pair of files is the following one:

workflow Mapping {
  String bin_path
  String sample_name
  String threads
  File human_g1k_v37_decoy_fa
  File human_g1k_v37_decoy_fai
  File human_g1k_v37_decoy_bwt
  File human_g1k_v37_decoy_pac
  File human_g1k_v37_decoy_ann
  File human_g1k_v37_decoy_amb
  File human_g1k_v37_decoy_sa

  call bwa1 {
    input: sample_name=sample_name, threads=threads, human_g1k_v37_decoy_fa=human_g1k_v37_decoy_fa, human_g1k_v37_decoy_fai=human_g1k_v37_decoy_fai, human_g1k_v37_decoy_bwt=human_g1k_v37_decoy_bwt, human_g1k_v37_decoy_pac=human_g1k_v37_decoy_pac, human_g1k_v37_decoy_ann=human_g1k_v37_decoy_ann, human_g1k_v37_decoy_amb=human_g1k_v37_decoy_amb, human_g1k_v37_decoy_sa=human_g1k_v37_decoy_sa
     }

  call bwa2 {
    input: sample_name=sample_name, threads=threads, human_g1k_v37_decoy_fa=human_g1k_v37_decoy_fa, human_g1k_v37_decoy_fai=human_g1k_v37_decoy_fai, human_g1k_v37_decoy_bwt=human_g1k_v37_decoy_bwt, human_g1k_v37_decoy_pac=human_g1k_v37_decoy_pac, human_g1k_v37_decoy_ann=human_g1k_v37_decoy_ann, human_g1k_v37_decoy_amb=human_g1k_v37_decoy_amb, human_g1k_v37_decoy_sa=human_g1k_v37_decoy_sa
    }


   call mergesamfiles {
    input: bin_path=bin_path, sample_name=sample_name, in1=bwa1.out, in2=bwa2.out
   }

  }

  task bwa1 {
  String sample_name
  String threads
  File human_g1k_v37_decoy_fa
  File human_g1k_v37_decoy_fai
  File human_g1k_v37_decoy_bwt
  File human_g1k_v37_decoy_pac
  File human_g1k_v37_decoy_ann
  File human_g1k_v37_decoy_amb
  File human_g1k_v37_decoy_sa
  String run
  String ID
  String SM
  String PL
  String LB
  String PU
  File R1
  File R2
  command {bwa mem -M -t ${threads} -R '@RG\tID:${ID}\tSM:${SM}\tPL:${PL}\tLB:${LB}\tPU:${PU}' ${human_g1k_v37_decoy_fa} ${R1} ${R2} > ${sample_name}_${run}.sam}
  output {File out ="${sample_name}_${run}.sam}"
    }
  }
  task bwa2 {
  String sample_name
  String threads
  File human_g1k_v37_decoy_fa
  File human_g1k_v37_decoy_fai
  File human_g1k_v37_decoy_bwt
  File human_g1k_v37_decoy_pac
  File human_g1k_v37_decoy_ann
  File human_g1k_v37_decoy_amb
  File human_g1k_v37_decoy_sa
  String run
  String ID
  String SM
  String PL
  String LB
  String PU
  File R1
  File R2
  command {bwa mem -M -t ${threads} -R '@RG\tID:${ID}\tSM:${SM}\tPL:${PL}\tLB:${LB}\tPU:${PU}' ${human_g1k_v37_decoy_fa} ${R1} ${R2} > ${sample_name}_${run}.sam}
  output {File out ="${sample_name}_${run}.sam}"
  }
 }

 task mergesamfiles{
  String bin_path
  String sample_name
  File in1
  File in2
  command {java -jar ${bin_path}/picard.jar MergeSamFiles I=${in1} I=${in2} O=${sample_name}_merged.sam}
  output {File out = "${sample_name}_merged.sam"}
 }

And the full error message I get when I run this wdl script:

[2017-09-07 07:54:44,06] [info] Running with database db.url = jdbc:hsqldb:mem:81f5f1ca-cce2-443b-8ce9-4b0259c61812;shutdown=false;hsqldb.tx=mvcc
[2017-09-07 07:54:50,27] [info] Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000
[2017-09-07 07:54:50,28] [info] [RenameWorkflowOptionsInMetadata] 100%
[2017-09-07 07:54:50,63] [info] Slf4jLogger started
[2017-09-07 07:54:50,87] [info] Metadata summary refreshing every 2 seconds.
[2017-09-07 07:54:50,93] [info] CallCacheWriteActor configured to write to the database with batch size 100 and flush rate 3 seconds.
[2017-09-07 07:54:50,95] [info] WriteMetadataActor configured to write to the database with batch size 200 and flush rate 5 seconds.
[2017-09-07 07:54:51,76] [info] SingleWorkflowRunnerActor: Submitting workflow
[2017-09-07 07:54:51,79] [info] Workflow c7822e7c-6361-4aa9-b7f8-0483798baf72 submitted.
[2017-09-07 07:54:51,80] [info] SingleWorkflowRunnerActor: Workflow submitted c7822e7c-6361-4aa9-b7f8-0483798baf72
[2017-09-07 07:54:51,80] [info] 1 new workflows fetched
[2017-09-07 07:54:51,80] [info] WorkflowManagerActor Starting workflow c7822e7c-6361-4aa9-b7f8-0483798baf72
[2017-09-07 07:54:51,81] [info] WorkflowManagerActor Successfully started WorkflowActor-c7822e7c-6361-4aa9-b7f8-0483798baf72
[2017-09-07 07:54:51,81] [info] Retrieved 1 workflows from the WorkflowStoreActor
[2017-09-07 07:54:52,11] [info] MaterializeWorkflowDescriptorActor [c7822e7c]: Call-to-Backend assignments: Mapping.bwa2 -> Local, Mapping.mergesamfiles -> Local, Mapping.bwa1 -> Local
[2017-09-07 07:54:54,32] [info] WorkflowExecutionActor-c7822e7c-6361-4aa9-b7f8-0483798baf72 [c7822e7c]: Starting calls: Mapping.bwa1:NA:1, Mapping.bwa2:NA:1
[2017-09-07 07:54:54,57] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa2:NA:1]: bwa mem -M -t 4 -R '@RG\tID:1345.1515A05.1\tSM:1345\tPL:illumina\tLB:1345\tPU:1345' /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa2/inputs/mnt/Test_reference/human_g1k_v37_decoy.fasta /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa2/inputs/mnt/Test_151503/1345_TTAACGTC_L002_R1_001.fastq.gz /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa2/inputs/mnt/Test_151503/1345_TTAACGTC_L002_R2_001.fastq.gz > S1345_002.sam
[2017-09-07 07:54:54,57] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa1:NA:1]: bwa mem -M -t 4 -R '@RG\tID:1345.1515A05.1\tSM:1345\tPL:illumina\tLB:1345\tPU:1345' /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa1/inputs/mnt/Test_reference/human_g1k_v37_decoy.fasta /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa1/inputs/mnt/Test_151503/1345_TTAACGTC_L001_R1_001.fastq.gz /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa1/inputs/mnt/Test_151503/1345_TTAACGTC_L001_R2_001.fastq.gz > S1345_001.sam
[2017-09-07 07:54:54,59] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa2:NA:1]: executing: /bin/bash /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa2/execution/script
[2017-09-07 07:54:54,59] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa1:NA:1]: executing: /bin/bash /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa1/execution/script
[2017-09-07 07:54:54,65] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa1:NA:1]: job id: 3128
[2017-09-07 07:54:54,65] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa2:NA:1]: job id: 3130
[2017-09-07 07:54:54,66] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa1:NA:1]: Status change from - to WaitingForReturnCodeFile
[2017-09-07 07:54:54,66] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa2:NA:1]: Status change from - to WaitingForReturnCodeFile
[2017-09-07 07:56:23,29] [info] BackgroundConfigAsyncJobExecutionActor [c7822e7cMapping.bwa1:NA:1]: Status change from WaitingForReturnCodeFile to Done
[2017-09-07 07:56:23,73] [error] WorkflowManagerActor Workflow c7822e7c-6361-4aa9-b7f8-0483798baf72 failed (during ExecutingWorkflowState): Could not evaluate bwa1.out = "${sample_name}_${run}.sam}"
java.lang.RuntimeException: Could not evaluate bwa1.out = "${sample_name}_${run}.sam}"
        at wdl4s.wdl.WdlTask$$anonfun$4.applyOrElse(WdlTask.scala:189)
        at wdl4s.wdl.WdlTask$$anonfun$4.applyOrElse(WdlTask.scala:188)
        at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
        at scala.util.Failure.recoverWith(Try.scala:232)
        at wdl4s.wdl.WdlTask.$anonfun$evaluateOutputs$2(WdlTask.scala:188)
        at scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:157)
        at scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:157)
        at scala.collection.Iterator.foreach(Iterator.scala:929)
        at scala.collection.Iterator.foreach$(Iterator.scala:929)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
        at scala.collection.IterableLike.foreach(IterableLike.scala:71)
        at scala.collection.IterableLike.foreach$(IterableLike.scala:70)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:157)
        at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:155)
        at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
        at wdl4s.wdl.WdlTask.evaluateOutputs(WdlTask.scala:181)
        at cromwell.backend.wdl.OutputEvaluator$.evaluateOutputs(OutputEvaluator.scala:15)
        at cromwell.backend.standard.StandardAsyncExecutionActor.evaluateOutputs(StandardAsyncExecutionActor.scala:406)
        at cromwell.backend.standard.StandardAsyncExecutionActor.evaluateOutputs$(StandardAsyncExecutionActor.scala:405)
        at cromwell.backend.impl.sfs.config.BackgroundConfigAsyncJobExecutionActor.evaluateOutputs(ConfigAsyncJobExecutionActor.scala:121)
        at cromwell.backend.standard.StandardAsyncExecutionActor.handleExecutionSuccess(StandardAsyncExecutionActor.scala:458)
        at cromwell.backend.standard.StandardAsyncExecutionActor.handleExecutionSuccess$(StandardAsyncExecutionActor.scala:455)
        at cromwell.backend.impl.sfs.config.BackgroundConfigAsyncJobExecutionActor.handleExecutionSuccess(ConfigAsyncJobExecutionActor.scala:121)
        at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$handleExecutionResult$3(StandardAsyncExecutionActor.scala:651)
        at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:302)
        at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:37)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
        at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
        at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
        at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:38)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:43)
        at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.RuntimeException: Could not process output, file not found: /mnt/Test_151503/cromwell-executions/Mapping/c7822e7c-6361-4aa9-b7f8-0483798baf72/call-bwa1/execution/S1345_001.sam}
        at cromwell.backend.sfs.SharedFileSystem.mapJobWdlFile(SharedFileSystem.scala:144)
        at cromwell.backend.sfs.SharedFileSystem.mapJobWdlFile$(SharedFileSystem.scala:140)
        at cromwell.backend.sfs.SharedFileSystemJobCachingActorHelper$$anon$1.mapJobWdlFile(SharedFileSystemJobCachingActorHelper.scala:13)
        at cromwell.backend.sfs.SharedFileSystemAsyncJobExecutionActor.mapOutputWdlFile(SharedFileSystemAsyncJobExecutionActor.scala:213)
        at cromwell.backend.sfs.SharedFileSystemAsyncJobExecutionActor.mapOutputWdlFile$(SharedFileSystemAsyncJobExecutionActor.scala:212)
        at cromwell.backend.impl.sfs.config.BackgroundConfigAsyncJobExecutionActor.mapOutputWdlFile(ConfigAsyncJobExecutionActor.scala:121)
        at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$outputValueMapper$1(StandardAsyncExecutionActor.scala:387)
        at cromwell.backend.wdl.WdlFileMapper$.$anonfun$mapWdlFiles$1(WdlFileMapper.scala:11)
        at scala.util.Try$.apply(Try.scala:209)
        at cromwell.backend.wdl.WdlFileMapper$.mapWdlFiles(WdlFileMapper.scala:11)
        at cromwell.backend.standard.StandardAsyncExecutionActor.outputValueMapper(StandardAsyncExecutionActor.scala:387)
        at cromwell.backend.standard.StandardAsyncExecutionActor.outputValueMapper$(StandardAsyncExecutionActor.scala:386)
        at cromwell.backend.impl.sfs.config.BackgroundConfigAsyncJobExecutionActor.outputValueMapper(ConfigAsyncJobExecutionActor.scala:121)
        at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$evaluateOutputs$1(StandardAsyncExecutionActor.scala:406)
        at scala.util.Success.flatMap(Try.scala:247)
        ... 35 more

[2017-09-07 07:56:23,77] [warn] Couldn't find a suitable DSN, defaulting to a Noop one.
[2017-09-07 07:56:23,79] [info] Using noop to send events.
[2017-09-07 07:56:23,84] [info] WorkflowManagerActor WorkflowActor-c7822e7c-6361-4aa9-b7f8-0483798baf72 is in a terminal state: WorkflowFailedState
[2017-09-07 07:56:34,85] [info] Message [cromwell.backend.async.AsyncBackendJobExecutionActor$IssuePollRequest] from Actor[akka://cromwell-system/user/SingleWorkflowRunnerActor/WorkflowManagerActor/WorkflowActor-c7822e7c-6361-4aa9-b7f8-0483798baf72/WorkflowExecutionActor-c7822e7c-6361-4aa9-b7f8-0483798baf72/c7822e7c-6361-4aa9-b7f8-0483798baf72-EngineJobExecutionActor-Mapping.bwa2:NA:1/c7822e7c-6361-4aa9-b7f8-0483798baf72-BackendJobExecutionActor-c7822e7c:Mapping.bwa2:-1:1/BackgroundConfigAsyncJobExecutionActor#1467712241] to Actor[akka://cromwell-system/user/SingleWorkflowRunnerActor/WorkflowManagerActor/WorkflowActor-c7822e7c-6361-4aa9-b7f8-0483798baf72/WorkflowExecutionActor-c7822e7c-6361-4aa9-b7f8-0483798baf72/c7822e7c-6361-4aa9-b7f8-0483798baf72-EngineJobExecutionActor-Mapping.bwa2:NA:1/c7822e7c-6361-4aa9-b7f8-0483798baf72-BackendJobExecutionActor-c7822e7c:Mapping.bwa2:-1:1/BackgroundConfigAsyncJobExecutionActor#1467712241] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[2017-09-07 07:56:41,40] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.
Workflow c7822e7c-6361-4aa9-b7f8-0483798baf72 transitioned to state Failed
[2017-09-07 07:56:41,52] [info] Automatic shutdown of the async connection
[2017-09-07 07:56:41,52] [info] Gracefully shutdown sentry threads.
[2017-09-07 07:56:41,52] [info] Shutdown finished.

Can someone help me solve this issue? The thing is that the two sam files that serve as input files for the last command are actually being generated and they are in their respective subfolders. I have seen similar posts in the last few days, but I haven't found the answer to my question in them.

Just in case anyone comments, I know I could scatter the 8 runs in a single block code, but I haven't reach this point so far.

Thank you very much,

Yatros

Best Answer

Answers

Sign In or Register to comment.