Can't Localize File in Google Storage Bucket (No URL)

amr@broadinstitute.orge[email protected] Member, Broadie
edited May 1 in Ask the WDL team

I have a task in a WDL that takes as input a GVCF file located in a bucket. The task fails to start with this error:

Failures: [{"message": "Workflow failed"
"causedBy": [{"message": "Task gatk_process_cohort.gatk_joint_genotype_task:NA:1 failed. The job was stopped before the command finished. PAPI error code 5. Message: 9: Failed to localize files: failed to copy the following files: \"gs://broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf\\t\\r -> /mnt/local-disk/broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf\\t\\r (cp failed: gsutil -q -m cp gs://broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf\\t\\r /mnt/local-disk/broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf\\t\\r
command failed: CommandException: No URLs matched: gs://broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf\\t\\r\\nCommandException: 1 file/object could not be trans ferred.\\n)\""

The file in question, 'gs://broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf' does exist in this location as seen by a gsutil ls command:

12:28pm [email protected] ~ $ gsutil ls gs://broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf
gs://broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf
9.125u 1.536s 0:12.65 84.1%     0+0k 16+40io 2pf+0w
12:28pm [email protected] ~ $ 

So I'm at a loss as to why this error is occurring. Any help appreciated!

Best Answer

  • amr@broadinstitute.orge[email protected] Member, Broadie
    Accepted Answer

    Just thought I'd close this out by mentioning the problem was that the GS URL that was causing the error had hidden characters in it after the file extension:
    gs://broad-cil-devel-bucket/broad-file-inputs/G:/Candida_Auris.gvcf\t\r

    These were being added by the CLI we use to launch our workflows, and was resolved by fixing a bug in the CLI code.

Answers

Sign In or Register to comment.