Latest Release: 05/01/19
Release Notes can be found here.

Firecloud submits duplicate jobs within a pair_set

Hi Firecloud Team,

this is about: broad-firecloud-testing/SKCM_PanCan

I ran an analysis on a pair set of exactly 100 pairs. When I hit launch, it showed 100 jobs running. This is what I expected. (expression: this.pairs)

Outputs of this analysis are named ${pair_id}-sim-final.acs.seg

Once the workflow completed, the status of each job went to "Unknown", but indiviudally they all succeeded. As a result no annotations were written, and I made a separate post about that issue, but it is relevant to this post as well.

In order to get the annotations, I ran

gsutil ls "gs://fc-010c62db-aa35-4b0e-b6f4-eec18cc585fc/ee83b780-33e8-4f20-8794-74bf48947cf0/**.acs.seg"

This returned 106 lines.

Several samples were duplicated, but the paths were unique, for example:

gs://fc-010c62db-aa35-4b0e-b6f4-eec18cc585fc/ee83b780-33e8-4f20-8794-74bf48947cf0/gatk_acnv_wkfl/3e22f242-126c-4e50-b280-ba31854988aa/call-gatk_acnv/acnv/TCGA-D3-A3CB-06A-11D_TCGA-D3-A3CB-10A-01D-sim-final.acs.seg
gs://fc-010c62db-aa35-4b0e-b6f4-eec18cc585fc/ee83b780-33e8-4f20-8794-74bf48947cf0/gatk_acnv_wkfl/f7094268-f03e-4182-84a2-b46cc8bdaef2/call-gatk_acnv/acnv/TCGA-D3-A3CB-06A-11D_TCGA-D3-A3CB-10A-01D-sim-final.acs.seg

As you can see, they were part of the same submission ID, but got unique workflows even though they were from the same pair.

I have never counted the outputs in a workflow before, so I do not know if this is a recurring issue.

Best Answer

Answers

Sign In or Register to comment.