Heads up:
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.

How to launch a batch of sample analysis in Firecloud?

Here I have TCGA controlled data access authorization, and I have successfully run some sample (sample entity, use the bam files) analysis in FireCloud.

However, I want to run hundreds of analysis and I can launch the analysis one-by-one in a hard way. It's really time consuming. I've also try to use the "sample_set" Root Entity Type, but IT DIDN"T WORK. I wonder if these is some other ways to launch the analysis in a batch, not one-by-one? Thank you.

Best Answer

Answers

  • jemimalwhjemimalwh ChinaMember

    "Similarly, if the method ran on a sample and you selected a sample set, “this” refers to each sample in the sample set. In this case, one workflow would be created for each entity in the set and submitted all at once." (FireCloud Basis https://docs.google.com/document/d/1X7q4zYAb16Py8raxGhP_HPzp5KRjrNfTeSR0wIRrzQU/edit#)

    The root entity type of my method is "sample". And this method works when launch on single sample ("this" refers to the sample). But when I try to launch a sample_set, it failed with the following ERROR message:

    "Error 400: Method configuration expects an entity of type sample, but you gave us an entity of type sample_set.
    Occurred: July 27, 2016 9:45 AM +08:00
    Source: rawls"

  • jneffjneff BostonMember, Broadie admin

    When launching an analysis, if you select a sample_set, you can then enter "this.samples" in the Define Expression field. See also Workflow Expansion Expressions in the documentation.

    I will update the examples to include sample_set.

  • jemimalwhjemimalwh ChinaMember

    @jneff said:
    When launching an analysis, if you select a sample_set, you can then enter "this.samples" in the Define Expression field. See also Workflow Expansion Expressions in the documentation.

    I will update the examples to include sample_set.

    I have also read the "Workflow Expansion Expression", and I have also tried, but it doesn't work.

    First try:

    1) "Root Entity Type" == "sample"
    2) inputs: this.samples.attribut, this.samples.name
    3) outputs: this.output1, this.output2
    Error message:"Error 400: Method configuration expects an entity of type sample, but you gave us an entity of type sample_set."

    Second try:

    1) "Root Entity Type" == "sample_set"
    2) inputs: this.samples.attribut, this.samples.name
    3) outputs: this.output1, this.output2
    Error message: "Expected single value for workflow input, but evaluated result set had multiple values."

  • abaumannabaumann Broad DSDEMember, Broadie ✭✭✭

    Actually I just realized something was wrong in your first try: your expressions for inputs should be "this.attribute" not "this.samples.attribute". This method is configured to run on a sample, so you define inputs and outputs relative to a single sample. The screenshot from above defines (at launch time) how to launch this workflow on each sample in the sample set.

  • jemimalwhjemimalwh ChinaMember

    @abaumann said:
    Your first try is what you want, the root entity type is what you want the method to be run on, which is a sample. You just want to run on each sample in a sample set, which requires you to define an expression at launch time (not in the method configuration).

    In the launch analysis dialog, click on the sample set you want to use. This will make the box that says "Define Expression" editable. This expression lets you effectively turn one type into another. For instance to turn a sample set into a workflow run for each sample in that sample set you enter "this.samples".

    I have attached a screenshot that will hopefully clarify. Sorry this is so complicated - we will try to simplify the interface!

    It works! Thank you!

  • abaumannabaumann Broad DSDEMember, Broadie ✭✭✭
Sign In or Register to comment.