ERROR: Finished parsing without consuming all tokens.

Trying to follow example here: http://gatkforums.broadinstitute.org/wdl/discussion/7221/2-howto-write-a-simple-multi-step-workflow

I editted the json file (simpleVariantSelection_inputs.json) with the full paths on my local machine and replacing the "File" and "String" placeholders. However when I run I am getting the following error:

[2017-05-25 12:18:24,85] [info] WorkflowManagerActor Successfully started WorkflowActor-e52409b4-c85a-4285-9453-f47c6b0ae86c
[2017-05-25 12:18:24,85] [info] Retrieved 1 workflows from the WorkflowStoreActor
[2017-05-25 12:18:24,93] [error] WorkflowManagerActor Workflow e52409b4-c85a-4285-9453-f47c6b0ae86c failed (during MaterializingWorkflowDescriptorState): Workflow input processing failed:
Unable to load namespace from workflow: ERROR: Finished parsing without consuming all tokens.

{
^

cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor$$anonfun$3$$anon$1: Workflow input processing failed:
Unable to load namespace from workflow: ERROR: Finished parsing without consuming all tokens.

{
^

at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor$$anonfun$3.applyOrElse(MaterializeWorkflowDescriptorActor.scala:137)
at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor$$anonfun$3.applyOrElse(MaterializeWorkflowDescriptorActor.scala:129)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
at akka.actor.FSM$class.processEvent(FSM.scala:663)
at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor.akka$actor$LoggingFSM$$super$processEvent(MaterializeWorkflowDescriptorActor.scala:116)
at akka.actor.LoggingFSM$class.processEvent(FSM.scala:799)
at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor.processEvent(MaterializeWorkflowDescriptorActor.scala:116)
at akka.actor.FSM$class.akka$actor$FSM$$processMsg(FSM.scala:657)
at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:651)
at akka.actor.Actor$class.aroundReceive(Actor.scala:496)
at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor.aroundReceive(MaterializeWorkflowDescriptorActor.scala:116)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[2017-05-25 12:18:24,94] [info] WorkflowManagerActor WorkflowActor-e52409b4-c85a-4285-9453-f47c6b0ae86c is in a terminal state: WorkflowFailedState
[2017-05-25 12:18:24,94] [info] Message [cromwell.subworkflowstore.SubWorkflowStoreActor$SubWorkflowStoreCompleteSuccess] from Actor[akka://cromwell-system/user/SingleWorkflowRunnerActor/$c#-297741123] to Actor[akka://cromwell-system/user/SingleWorkflowRunnerActor/WorkflowManagerActor/WorkflowActor-e52409b4-c85a-4285-9453-f47c6b0ae86c#772660809] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[2017-05-25 12:18:26,88] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.
Workflow e52409b4-c85a-4285-9453-f47c6b0ae86c transitioned to state Failed

I recognize the brackets are the problem, but the tutorial doesn't seem to offer suggestions on removing those. When I did remove them it seemed the key values in the json were the problem. Any help is greatly appreciated!

Tagged:

Best Answer

Answers

  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev

    Hmm, that error message isn't enormously helpful for debugging, for sure!

    I'd suggest copying the input JSON into a syntax validator to highlight errors for you better, for example you could try jsonlint.

    If that doesn't help and you're happy to, you can copy the inputs JSON here and I can have a closer look.

    Hope that helps!

  • afriedenafrieden Cambridge, MAMember

    Ya sure thing. Initially I ran to create to the json file:

    java -jar /Users/alexanderfrieden/projects/wdltool/target/scala-2.12/wdltool-0.11.jar inputs ~/Downloads/SimpleVariantSelection/simpleVariantSelection.wdl > simpleVariantSelection_inputs.json
    

    And then manually edited the file. Below is the full file.

    "SimpleVariantSelection.refIndex": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta.fai",
    "SimpleVariantSelection.name": "NA12878",
    "SimpleVariantSelection.refDict": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta.dict",
    "SimpleVariantSelection.haplotypeCaller.bamIndex": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/inputs/NA12878_wgs_20.bai",
    "SimpleVariantSelection.refFasta": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta",
    "SimpleVariantSelection.gatk": "/Users/alexanderfrieden/Downloads/GenomeAnalysisTK-3.7/GenomeAnalysisTK.jar",
    "SimpleVariantSelection.haplotypeCaller.inputBAM": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/inputs/NA12878_wgs_20.bam"
    

    Let me know what I should be trying. Thanks!

  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev

    I think you do need the curly braces, since the input file should be a JSON object. I ran this through the JSON validator and it was accepted as valid JSON. I'd be interested in seeing whether Cromwell accepts it now:

    {
        "SimpleVariantSelection.refIndex": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta.fai",
        "SimpleVariantSelection.name": "NA12878",
        "SimpleVariantSelection.refDict": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta.dict",
        "SimpleVariantSelection.haplotypeCaller.bamIndex": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/inputs/NA12878_wgs_20.bai",
        "SimpleVariantSelection.refFasta": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta",
        "SimpleVariantSelection.gatk": "/Users/alexanderfrieden/Downloads/GenomeAnalysisTK-3.7/GenomeAnalysisTK.jar",
        "SimpleVariantSelection.haplotypeCaller.inputBAM": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/inputs/NA12878_wgs_20.bam"
    }
    
  • afriedenafrieden Cambridge, MAMember

    This produces the same error as in the initial post. Removing them creates the error of key values.

    Command I am running is:

    java -jar /usr/local/Cellar/cromwell/26/libexec/cromwell-26.jar run simpleVariantSelection_inputs.json ~/Downloads/SimpleVariantSelection/simpleVariantSelection.wdl
    
  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev

    OK, I actually think the input JSON is correct and it's instead objecting to a syntax error in the WDL.

    Could you copy in the WDL file you're using (if you copy/pasted from the website, perhaps you've missed out an opening { or a closing }?)

    Out of interest, which version of Cromwell are you using? I know a lot of the error reporting was improved in C27 but perhaps this is another case to improve (we have quite a few obscure error messages!).

  • afriedenafrieden Cambridge, MAMember

    WDL file:

        workflow SimpleVariantSelection {
        File gatk
        File refFasta
        File refIndex
        File refDict    
        String name
    
        call haplotypeCaller {
              input: sampleName=name,
                          RefFasta=refFasta,
                          GATK=gatk,
                          RefIndex=refIndex,
                          RefDict=refDict
        }
        call select as selectSNPs {
              input: sampleName=name,
                          RefFasta=refFasta,
                          GATK=gatk,
                          RefIndex=refIndex,
                          RefDict=refDict,
                          type="SNP",
                          rawVCF=haplotypeCaller.rawVCF
        }
        call select as selectIndels {
              input: sampleName=name,
                          RefFasta=refFasta,
                          GATK=gatk,
                          RefIndex=refIndex,
                          RefDict=refDict,
                          type="INDEL",
                          rawVCF=haplotypeCaller.rawVCF
        }
    }
    
    task haplotypeCaller {
      File GATK
      File RefFasta
      File RefIndex
      File RefDict
      String sampleName
      File inputBAM
      File bamIndex
    
      command {
        java -jar ${GATK} \
          -T HaplotypeCaller \
          -R ${RefFasta} \
          -I ${inputBAM} \
          -o ${sampleName}.raw.indels.snps.vcf
      }
      output {
        File rawVCF = "${sampleName}.raw.indels.snps.vcf"
      }
    }
    
    task select {
      File GATK
      File RefFasta
      File RefIndex
      File RefDict
      String sampleName
      String type
      File rawVCF
    
      command {
        java -jar ${GATK} \
          -T SelectVariants \
          -R ${RefFasta} \
          -V ${rawVCF} \
          -selectType ${type} \
          -o ${sampleName}_raw.${type}.vcf
      }
      output {
        File rawSubset = "${sampleName}_raw.${type}.vcf"
      }
    }
    
Sign In or Register to comment.