To celebrate the release of GATK 4.0, we are giving away free credits for running the GATK4 Best Practices pipelines in FireCloud, our secure online analysis portal. It’s first come first serve, so sign up now to claim your free credits worth $250. Sponsored by Google Cloud. Learn more at https://software.broadinstitute.org/firecloud/documentation/freecredits

ERROR: Finished parsing without consuming all tokens.

afriedenafrieden Cambridge, MAMember

Trying to follow example here: http://gatkforums.broadinstitute.org/wdl/discussion/7221/2-howto-write-a-simple-multi-step-workflow

I editted the json file (simpleVariantSelection_inputs.json) with the full paths on my local machine and replacing the "File" and "String" placeholders. However when I run I am getting the following error:

[2017-05-25 12:18:24,85] [info] WorkflowManagerActor Successfully started WorkflowActor-e52409b4-c85a-4285-9453-f47c6b0ae86c
[2017-05-25 12:18:24,85] [info] Retrieved 1 workflows from the WorkflowStoreActor
[2017-05-25 12:18:24,93] [error] WorkflowManagerActor Workflow e52409b4-c85a-4285-9453-f47c6b0ae86c failed (during MaterializingWorkflowDescriptorState): Workflow input processing failed:
Unable to load namespace from workflow: ERROR: Finished parsing without consuming all tokens.

{
^

cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor$$anonfun$3$$anon$1: Workflow input processing failed:
Unable to load namespace from workflow: ERROR: Finished parsing without consuming all tokens.

{
^

at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor$$anonfun$3.applyOrElse(MaterializeWorkflowDescriptorActor.scala:137)
at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor$$anonfun$3.applyOrElse(MaterializeWorkflowDescriptorActor.scala:129)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36)
at akka.actor.FSM$class.processEvent(FSM.scala:663)
at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor.akka$actor$LoggingFSM$$super$processEvent(MaterializeWorkflowDescriptorActor.scala:116)
at akka.actor.LoggingFSM$class.processEvent(FSM.scala:799)
at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor.processEvent(MaterializeWorkflowDescriptorActor.scala:116)
at akka.actor.FSM$class.akka$actor$FSM$$processMsg(FSM.scala:657)
at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:651)
at akka.actor.Actor$class.aroundReceive(Actor.scala:496)
at cromwell.engine.workflow.lifecycle.MaterializeWorkflowDescriptorActor.aroundReceive(MaterializeWorkflowDescriptorActor.scala:116)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[2017-05-25 12:18:24,94] [info] WorkflowManagerActor WorkflowActor-e52409b4-c85a-4285-9453-f47c6b0ae86c is in a terminal state: WorkflowFailedState
[2017-05-25 12:18:24,94] [info] Message [cromwell.subworkflowstore.SubWorkflowStoreActor$SubWorkflowStoreCompleteSuccess] from Actor[akka://cromwell-system/user/SingleWorkflowRunnerActor/$c#-297741123] to Actor[akka://cromwell-system/user/SingleWorkflowRunnerActor/WorkflowManagerActor/WorkflowActor-e52409b4-c85a-4285-9453-f47c6b0ae86c#772660809] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'.
[2017-05-25 12:18:26,88] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.
Workflow e52409b4-c85a-4285-9453-f47c6b0ae86c transitioned to state Failed

I recognize the brackets are the problem, but the tutorial doesn't seem to offer suggestions on removing those. When I did remove them it seemed the key values in the json were the problem. Any help is greatly appreciated!

Tagged:

Best Answer

Answers

  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev

    Hmm, that error message isn't enormously helpful for debugging, for sure!

    I'd suggest copying the input JSON into a syntax validator to highlight errors for you better, for example you could try jsonlint.

    If that doesn't help and you're happy to, you can copy the inputs JSON here and I can have a closer look.

    Hope that helps!

  • afriedenafrieden Cambridge, MAMember

    Ya sure thing. Initially I ran to create to the json file:

    java -jar /Users/alexanderfrieden/projects/wdltool/target/scala-2.12/wdltool-0.11.jar inputs ~/Downloads/SimpleVariantSelection/simpleVariantSelection.wdl > simpleVariantSelection_inputs.json
    

    And then manually edited the file. Below is the full file.

    "SimpleVariantSelection.refIndex": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta.fai",
    "SimpleVariantSelection.name": "NA12878",
    "SimpleVariantSelection.refDict": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta.dict",
    "SimpleVariantSelection.haplotypeCaller.bamIndex": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/inputs/NA12878_wgs_20.bai",
    "SimpleVariantSelection.refFasta": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta",
    "SimpleVariantSelection.gatk": "/Users/alexanderfrieden/Downloads/GenomeAnalysisTK-3.7/GenomeAnalysisTK.jar",
    "SimpleVariantSelection.haplotypeCaller.inputBAM": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/inputs/NA12878_wgs_20.bam"
    

    Let me know what I should be trying. Thanks!

  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev

    I think you do need the curly braces, since the input file should be a JSON object. I ran this through the JSON validator and it was accepted as valid JSON. I'd be interested in seeing whether Cromwell accepts it now:

    {
        "SimpleVariantSelection.refIndex": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta.fai",
        "SimpleVariantSelection.name": "NA12878",
        "SimpleVariantSelection.refDict": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta.dict",
        "SimpleVariantSelection.haplotypeCaller.bamIndex": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/inputs/NA12878_wgs_20.bai",
        "SimpleVariantSelection.refFasta": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/ref/human_g1k_b37_20.fasta",
        "SimpleVariantSelection.gatk": "/Users/alexanderfrieden/Downloads/GenomeAnalysisTK-3.7/GenomeAnalysisTK.jar",
        "SimpleVariantSelection.haplotypeCaller.inputBAM": "/Users/alexanderfrieden/Downloads/SimpleVariantSelection/inputs/NA12878_wgs_20.bam"
    }
    
  • afriedenafrieden Cambridge, MAMember

    This produces the same error as in the initial post. Removing them creates the error of key values.

    Command I am running is:

    java -jar /usr/local/Cellar/cromwell/26/libexec/cromwell-26.jar run simpleVariantSelection_inputs.json ~/Downloads/SimpleVariantSelection/simpleVariantSelection.wdl
    
  • ChrisLChrisL Cambridge, MAMember, Broadie, Moderator, Dev

    OK, I actually think the input JSON is correct and it's instead objecting to a syntax error in the WDL.

    Could you copy in the WDL file you're using (if you copy/pasted from the website, perhaps you've missed out an opening { or a closing }?)

    Out of interest, which version of Cromwell are you using? I know a lot of the error reporting was improved in C27 but perhaps this is another case to improve (we have quite a few obscure error messages!).

  • afriedenafrieden Cambridge, MAMember

    WDL file:

        workflow SimpleVariantSelection {
        File gatk
        File refFasta
        File refIndex
        File refDict    
        String name
    
        call haplotypeCaller {
              input: sampleName=name,
                          RefFasta=refFasta,
                          GATK=gatk,
                          RefIndex=refIndex,
                          RefDict=refDict
        }
        call select as selectSNPs {
              input: sampleName=name,
                          RefFasta=refFasta,
                          GATK=gatk,
                          RefIndex=refIndex,
                          RefDict=refDict,
                          type="SNP",
                          rawVCF=haplotypeCaller.rawVCF
        }
        call select as selectIndels {
              input: sampleName=name,
                          RefFasta=refFasta,
                          GATK=gatk,
                          RefIndex=refIndex,
                          RefDict=refDict,
                          type="INDEL",
                          rawVCF=haplotypeCaller.rawVCF
        }
    }
    
    task haplotypeCaller {
      File GATK
      File RefFasta
      File RefIndex
      File RefDict
      String sampleName
      File inputBAM
      File bamIndex
    
      command {
        java -jar ${GATK} \
          -T HaplotypeCaller \
          -R ${RefFasta} \
          -I ${inputBAM} \
          -o ${sampleName}.raw.indels.snps.vcf
      }
      output {
        File rawVCF = "${sampleName}.raw.indels.snps.vcf"
      }
    }
    
    task select {
      File GATK
      File RefFasta
      File RefIndex
      File RefDict
      String sampleName
      String type
      File rawVCF
    
      command {
        java -jar ${GATK} \
          -T SelectVariants \
          -R ${RefFasta} \
          -V ${rawVCF} \
          -selectType ${type} \
          -o ${sampleName}_raw.${type}.vcf
      }
      output {
        File rawSubset = "${sampleName}_raw.${type}.vcf"
      }
    }
    
Sign In or Register to comment.