Scala error on running new task

amaroamaro Broad InstituteMember, Broadie

What does this error mean? There is no stderr or stdout the only error I can find is from the log:

2017-06-06 17:36:09,014 INFO - MaterializeWorkflowDescriptorActor [UUID(000771a9)]: Call-to-Backend assignments: birdseed_to_vcf_wklfw.birdseed_to_vcf -> JES
2017-06-06 17:36:09,078 INFO - JES [UUID(000771a9)]: Creating authentication file for workflow 000771a9-f300-46a5-921e-3ad357319fb4 at
gs://cromwell-auth-vanallen-firecloud-billing/000771a9-f300-46a5-921e-3ad357319fb4_auth.json
2017-06-06 17:36:11,670 INFO - WorkflowExecutionActor-000771a9-f300-46a5-921e-3ad357319fb4 [UUID(000771a9)]: Starting calls: birdseed_to_vcf_wklfw.birdseed_to_vcf:NA:1
2017-06-06 17:36:11,877 ERROR - JesAsyncBackendJobExecutionActor
[UUID(000771a9)birdseed_to_vcf_wklfw.birdseed_to_vcf:NA:1]: Error attempting to Execute
scala.MatchError: null
at wdl4s.expression.ValueEvaluator.evaluate(ValueEvaluator.scala:45)
at wdl4s.expression.ValueEvaluator.evaluate(ValueEvaluator.scala:118)
at wdl4s.expression.ValueEvaluator.evaluate(ValueEvaluator.scala:118)
at wdl4s.expression.ValueEvaluator.evaluate(ValueEvaluator.scala:53)
at wdl4s.expression.ValueEvaluator.evaluate(ValueEvaluator.scala:52)
at wdl4s.WdlExpression$.evaluate(WdlExpression.scala:85)
at wdl4s.WdlExpression.evaluate(WdlExpression.scala:166)
at wdl4s.command.ParameterCommandPart.instantiate(ParameterCommandPart.scala:53)
at wdl4s.Task$$anonfun$instantiateCommand$1$$anonfun$apply$2.apply(Task.scala:108)
at wdl4s.Task$$anonfun$instantiateCommand$1$$anonfun$apply$2.apply(Task.scala:108)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at wdl4s.Task$$anonfun$instantiateCommand$1.apply(Task.scala:108)
at wdl4s.Task$$anonfun$instantiateCommand$1.apply(Task.scala:108)
at scala.util.Try$.apply(Try.scala:192)
at wdl4s.Task.instantiateCommand(Task.scala:108)
at cromwell.backend.wdl.Command$$anonfun$instantiate$1.apply(Command.scala:28)
at cromwell.backend.wdl.Command$$anonfun$instantiate$1.apply(Command.scala:27)
at scala.util.Success.flatMap(Try.scala:231)
at cromwell.backend.wdl.Command$.instantiate(Command.scala:27)
at cromwell.backend.standard.StandardAsyncExecutionActor$class.instantiatedCommand(StandardAsyncExecutionActor.scala:206)
at cromwell.backend.impl.jes.JesAsyncBackendJobExecutionActor.instantiatedCommand$lzycompute(JesAsyncBackendJobExecutionActor.scala:67)
at cromwell.backend.impl.jes.JesAsyncBackendJobExecutionActor.instantiatedCommand(JesAsyncBackendJobExecutionActor.scala:67)
at cromwell.backend.standard.StandardAsyncExecutionActor$class.commandScriptContents(StandardAsyncExecutionActor.scala:177)
at cromwell.backend.impl.jes.JesAsyncBackendJobExecutionActor.commandScriptContents(JesAsyncBackendJobExecutionActor.scala:67)
at cromwell.backend.impl.jes.JesAsyncBackendJobExecutionActor.cromwell$backend$impl$jes$JesAsyncBackendJobExecutionActor$$uploadScriptFile$1(JesAsyncBackendJobExecutionActor.scala:301)
at cromwell.backend.impl.jes.JesAsyncBackendJobExecutionActor$$anonfun$runWithJes$1$$anonfun$apply$3.apply(JesAsyncBackendJobExecutionActor.scala:315)
at cromwell.backend.impl.jes.JesAsyncBackendJobExecutionActor$$anonfun$runWithJes$1$$anonfun$apply$3.apply(JesAsyncBackendJobExecutionActor.scala:314)
at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:253)
at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:251)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2017-06-06 17:36:13,927 INFO - $h [UUID(000771a9)]: Copying workflow logs from /cromwell-workflow-logs/workflow.000771a9-f300-46a5-921e-3ad357319fb4.log to gs://fc-bb7e37f6-c15e-4f83-b468-23003a8ba3ff/d1294b07-55bc-4471-823b-d57c41566ce3/workflow.logs/workflow.000771a9-f300-46a5-921e-3ad357319fb4.log

Answers

  • esalinasesalinas ✭✭✭ BroadMember, Broadie ✭✭✭
    edited June 2017

    hi @amaro

    are you using regular expressions in the WDL anywhere. I think i've seen that error when there's a problem with a regex (no match?)

    -eddie

  • amaroamaro Broad InstituteMember, Broadie
    edited June 2017

    Hey
    @esalinas I guess one more upvote for better error reporting! I do have one regular expression. Here is the piece of the WDL the regular expression is in bold is parsing a File parameter. Is it because Im treating a File as a string? I think the regexp is correct.

      array_sample=${birdseed/.birdseed.txt/} 
      python BirdseedToVCF.py \
            --birdseed ${birdseed} \ 
            --output_vcf ${vcf_sample}.vcf \
            --snp_annotation_file ${snp_annotation_file} \
            --array_sample $array_sample \
            --vcf_sample ${vcf_sample} \
            --fasta ${ref_fasta}
    
  • esalinasesalinas ✭✭✭ BroadMember, Broadie ✭✭✭

    hi @amaro,

    I presume that the regex you refer to is replace the string "birdseed" with the string "birdseed.txt". Do I interpret that correctly?

    Second, what variable is being subjected to the regex search-and-replace? I don't use WDL regex much.

    Also, I think I've seen the "scala.MatchError: null" when the variable inside the "{....}" is not right. For example, I think I saw the error when the variable referenced was mispelled or "computed" incorrectly or something along that line.

    -eddie

  • amaroamaro Broad InstituteMember, Broadie

    hey @esalinas

    Thanks for the help. Instead of dealing with wdl (unclear how to diagnose errors) I moved all the logic into my actual script.. while not ideal it did get the job done.

  • esalinasesalinas ✭✭✭ BroadMember, Broadie ✭✭✭

    okay @amaro .... thanks for the update.

    -eddie

  • esalinasesalinas ✭✭✭ BroadMember, Broadie ✭✭✭

    @amaro after you removed the WDL regex do you confirm the "scala.MatchError: null" went away?

    -eddie

  • amaroamaro Broad InstituteMember, Broadie

    @esalinas yup it did! The actual regex statement worked perfectly well in bash though so Im confused why the wdl file caused an error.

  • ThibThib ✭✭ CambridgeMember, Broadie, Dev ✭✭

    Hi !

    Thanks for reporting this !

    I think there are a few things happening here.
    First you're right that the error message should be better and I just made a ticket to fix that.

    If I understand correctly, you're trying to use bash "replace" mechanism, like ${parameter/pattern/string} ?
    This will clash with WDL's syntax which happens to also use ${...} to insert WDL expressions in the command.
    I suppose you have an input File birdseed in your task ?
    If you want to manipulate the birdseed value I recommend using the sub function which supports regexes and / or the basename function

    Something like this should get you close to what you want:

        File birdseed
        command {
            array_sample=${sub(birdseed, "birdseed.txt", "")}
        }
    

    If you really want to use the bash syntax you could still do something not super clean like

        File birdseed
        String replace_command = "{birdseed/birdseed.txt/}"
        command {
            birdseed=${birdseed}
            array_sample=$${replace_command}
        }
    
Sign In or Register to comment.