For WDL questions, see the WDL specification and WDL docs.
For Cromwell questions, see the Cromwell docs and please post any issues on Github.
Localization via hard link has failed
I'm using cromwell 29 with docker, and I keep getting this error:
[2017-09-15 09:08:57,11] [warn] Localization via hard link has failed: /data/test/cromwell-executions/variantDiscovery/5bb6004a-dcf9-4459-84cf-7b7ecb35d960/call-Report/variantDiscoveryReport/bc64e2c9-fd27-429c-82b2-22458b63d9eb/call-plotBam/inputs/data/test/cromwell-executions/variantDiscovery/5bb6004a-dcf9-4459-84cf-7b7ecb35d960/call-dedup/shard-1/execution/102517-23.dedup.bam -> /data/test/cromwell-executions/variantDiscovery/a4726bef-8d45-4746-ac99-9cabf9dadd36/call-dedup/shard-1/execution/102517-23.dedup.bam: Operation not permitted
Because I am using docker, soft-linking is not possible, so cromwell keeps copying over all the files it needs. I also generate a report for every step of the analysis, so cromwell makes a copy of every single output file! Because of this localization problem, this effectively doubles the disk size of every analysis, and I cannot complete my analysis because I run out of harddisk space.
I found a similar issue on the forum here https://gatkforums.broadinstitute.org/wdl/discussion/9477/localization-via-hard-link-has-failed, but the solution is not very clear. I think EADG added a user to each docker image he uses? That's not an option for me since I do not control all the docker images I use (e.g the broadinstitute images for gatk and picard).
How can I resolve this issue? Creating a copy of every bam and fastq file every time it is needed is not acceptable in my situation, and I also don't want to stop using docker, for obvious reasons.