We've moved!
You can find our new documentation site and support forum for posting questions here.

reference to files of items in a pairset with one row empty causes "[]" (an empty array) to be made

esalinasesalinas BroadMember, Broadie ✭✭✭

Matt Leventhal has a WDL of root entity-type "pair set".

For each pair in the pair set. Matt wanted to refer to the files under a specific column. For the WDL input of datatype "Array[File]+" he put
in the method config "this.pairs.THE_COLUMN_NAME".

Upon running the WDL however, the input resolved to "[]" (an empty array).

Inspection of the data entity model showed that one of the pairs in the pair set had an empty value for that column.

After creating a pair-set without that pair, he re-ran the WDL and this time, the input resolved to an expected list of files. The number of pairs in the set is more than 50. And only one was missing a value in the column.

In the first case, would it be better, instead of evaluating to an empty array to a) have a message saying that one of the pairs is missing a file or b) simply "pick up" the files for the pairs that did have them and use those for inputs?

Best Answer


  • esalinasesalinas BroadMember, Broadie ✭✭✭

    @KateN thanks for the update

  • birgerbirger Member, Broadie, CGA-mod ✭✭✭

    This is similar, but not the same as a scenario that the GDAC team (@dheiman @vickyhorst @smeiers) run into. In their case, they want to run a correlation analysis in a sample set between two sample attributes. The root entity type is sample_set, and they specify as inputs this.samples.attribute1 and this.samples.attribute2. These inputs mappings resolve to arrays. If the two attributes are not fully populated across all samples in the sample set, then they get arrays of potentially different lengths. This causes an issue as there analysis assumes each sample occupies the same position in the array. To address this they have introduced the concept of a null file to service as a placeholder to fill in the gaps. It is odd that this.pairs appears to behave differently from this.samples.

  • gordon123gordon123 BroadMember, Broadie

    Here is a recent comment I made explaining how the GDAC uses null files in more detail. https://gatkforums.broadinstitute.org/firecloud/discussion/comment/39981#Comment_39981

Sign In or Register to comment.