Holiday Notice:
The Frontline Support team will be offline February 18 for President's Day but will be back February 19th. Thank you for your patience as we get to all of your questions!

Latest Release: 02/05/19
Release Notes can be found here.

Firecloud submits duplicate jobs within a pair_set

Hi Firecloud Team,

this is about: broad-firecloud-testing/SKCM_PanCan

I ran an analysis on a pair set of exactly 100 pairs. When I hit launch, it showed 100 jobs running. This is what I expected. (expression: this.pairs)

Outputs of this analysis are named ${pair_id}-sim-final.acs.seg

Once the workflow completed, the status of each job went to "Unknown", but indiviudally they all succeeded. As a result no annotations were written, and I made a separate post about that issue, but it is relevant to this post as well.

In order to get the annotations, I ran

gsutil ls "gs://fc-010c62db-aa35-4b0e-b6f4-eec18cc585fc/ee83b780-33e8-4f20-8794-74bf48947cf0/**.acs.seg"

This returned 106 lines.

Several samples were duplicated, but the paths were unique, for example:

gs://fc-010c62db-aa35-4b0e-b6f4-eec18cc585fc/ee83b780-33e8-4f20-8794-74bf48947cf0/gatk_acnv_wkfl/3e22f242-126c-4e50-b280-ba31854988aa/call-gatk_acnv/acnv/TCGA-D3-A3CB-06A-11D_TCGA-D3-A3CB-10A-01D-sim-final.acs.seg
gs://fc-010c62db-aa35-4b0e-b6f4-eec18cc585fc/ee83b780-33e8-4f20-8794-74bf48947cf0/gatk_acnv_wkfl/f7094268-f03e-4182-84a2-b46cc8bdaef2/call-gatk_acnv/acnv/TCGA-D3-A3CB-06A-11D_TCGA-D3-A3CB-10A-01D-sim-final.acs.seg

As you can see, they were part of the same submission ID, but got unique workflows even though they were from the same pair.

I have never counted the outputs in a workflow before, so I do not know if this is a recurring issue.

Best Answer

Answers

Sign In or Register to comment.