Heads up:
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.
Update: July 26, 2019
This section of the forum is now closed; we are working on a new support model for WDL that we will share here shortly. For Cromwell-specific issues, see the Cromwell docs and post questions on Github.

call caching error with softlink output files

When call-caching is enabled for a workflow that has soft-linked files as output for some of the tasks, cromwell will not produce any output.

task softlink {
    command {
        date >> outfile.txt
        ln -s outfile.txt datum
    output {
        File datum = "datum"

workflow wf {
    call softlink

As far as I can tell, it attempts to link the soft-linked output file into the root directory where cromwell is run, which fails. However, it will then still report the final outputs, even though the file is missing.

[email protected]~/devel/wdl$ cromwell run softlink.wdl 
[2017-01-23 07:52:51,20] [warn] Localization via hard link has failed: /home/redmar/devel/wdl/cromwell-executions/wf/dbfff823-ce16-4fa8-909f-6f8ee7c763fa/call-softlink/execution/datum -> /home/redmar/devel/wdl/outfile.txt
[2017-01-23 07:52:51,21] [warn] Localization via copy has failed: /home/redmar/devel/wdl/outfile.txt
[INFO] [01/23/2017 07:52:51.289] [cromwell-system-akka.dispatchers.engine-dispatcher-33] [akka://cromwell-system/user/SingleWorkflowRunnerActor/WorkflowManagerActor/WorkflowActor-dbfff823-ce16-4fa8-909f-6f8ee7c763fa/WorkflowExecutionActor-dbfff823-ce16-4fa8-909f-6f8ee7c763fa] WorkflowExecutionActor-dbfff823-ce16-4fa8-909f-6f8ee7c763fa [UUID(dbfff823)]: Workflow wf complete. Final Outputs:
  "wf_softlink_datum": "file:///home/redmar/devel/wdl/cromwell-executions/wf/dbfff823-ce16-4fa8-909f-6f8ee7c763fa/call-softlink/execution/datum"
[email protected]:~/devel/wdl$  cat /home/redmar/devel/wdl/cromwell-executions/wf/dbfff823-ce16-4fa8-909f-6f8ee7c763fa/call-softlink/execution/datum
cat: /home/redmar/devel/wdl/cromwell-executions/wf/dbfff823-ce16-4fa8-909f-6f8ee7c763fa/call-softlink/execution/datum: No such file or directory


Sign In or Register to comment.