We've moved!
For WDL questions, see the WDL specification and WDL docs.
For Cromwell questions, see the Cromwell docs and please post any issues on Github.

Task should fail if File output is not known in advance.

yfarjounyfarjoun Broad InstituteDev ✭✭✭

Hi,

I had problems when using the following task:


The problem is that while the task succeeds, but the output is the local name, and not the delocalized name. Now while this might work in local and SGE back-ends, it failes in GCS since the delocalization needs to know in advance what file will be delocalized.

I think that this should be invalid wdl, so that people who try out their wdl in local and sge backends will not have to debug this feature when moving to cloud. Basically, if the output is of type File, it needs to be known in advance, so if the expression throws an exception (as it must have here) before the task is run, the task cannot be run successfully and a helpful error message should be returned.

Answers

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Right, the valid way to express this in WDL would be to declare your output file up front. Are you saying you would like wdltool validate to throw an error on this?

    Btw I'm moving your post to the WDL forum based on that assumption, let me know if I'm wrong...

  • yfarjounyfarjoun Broad InstituteDev ✭✭✭

    I don't think that wdltool should fail to validate, I think that cromwell should fail the task when it finds that it doesn't know the name of a file it need to delocalize (since it cannot find that out later) currently the task succeeds, the next task fails.

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Ah, well if you want this to be considered invalid wdl then wdltool validate would fail, and as a direct consequence of the same underlying check, so would Cromwell.

    This is essentially an input validation request, right?

  • yfarjounyfarjoun Broad InstituteDev ✭✭✭

    I don't think it's possible to know in advance if the task can succeed. But upon the invocation of the task it needs to be known what the output file name is (if it's a file)

  • Geraldine_VdAuweraGeraldine_VdAuwera Cambridge, MAMember, Administrator, Broadie admin

    Okay. I'm just trying to understand what you want to see happen here. You say:

    if the expression throws an exception (as it must have here) before the task is run, the task cannot be run successfully and a helpful error message should be returned.

    Are you considering the different parts of the command block as different tasks? Because that's not what Cromwell sees. As far as it's concerned, the full command block is a single task, even if there are multiple parts to your command. So Cromwell can't tell you if the expression failed and therefore the rest of the command failed. It can only tell you your task failed and point you to the log output. Or are you looking for something else here?

Sign In or Register to comment.