Latest Release: 03/12/19
Release Notes can be found here.

PAPI error code 5. Message: 10: Failed to delocalize files: failed to copy the following files:

I keep receiving this error when attempting to run my WDL script. I have run the WDL locally without error using the same docker image. I have permission to the files in the google bucket. I believe it may be something with the docker image and trying to run it on FireCloud. It is an image I created that is currently on dockerhub at kdgosik/demuxlet. Would there be permission issues that can arise by using certain docker images?

There is no stdout or stderr messages written, even though there should be a message to stdout for the task being called. I have tried using the same docker image and just writing an echo message to stdout, without calling any additional commands, and it still gives me a similar error of failed to copy the following files: ...

Any help would be greatly appreciated!

Full error message:

Task DemuxletScatter.demuxlet:0:1 failed. The job was stopped before the command finished. PAPI error code 5. Message: 10: Failed to delocalize files: failed to copy the following files: "/mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.sing2 -> gs://fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/5e6085a5-f972-453a-9812-83a9b6c8ad9f/DemuxletScatter/ddb5a1a8-f0da-403d-bfb5-e01aebe132de/call-demuxlet/shard-0/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.sing2 (cp failed: gsutil -q -m cp -L /var/log/google-genomics/out.log /mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.sing2 gs://fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/5e6085a5-f972-453a-9812-83a9b6c8ad9f/DemuxletScatter/ddb5a1a8-f0da-403d-bfb5-e01aebe132de/call-demuxlet/shard-0/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.sing2, command failed: CommandException: No URLs matched: /mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.sing2\nCommandException: 1 file/object could not be transferred.\n); /mnt/local-disk/demuxlet-0-rc.txt -> gs://fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/5e6085a5-f972-453a-9812-83a9b6c8ad9f/DemuxletScatter/ddb5a1a8-f0da-403d-bfb5-e01aebe132de/call-demuxlet/shard-0/demuxlet-0-rc.txt (cp failed: gsutil -q -m cp -L /var/log/google-genomics/out.log /mnt/local-disk/demuxlet-0-rc.txt gs://fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/5e6085a5-f972-453a-9812-83a9b6c8ad9f/DemuxletScatter/ddb5a1a8-f0da-403d-bfb5-e01aebe132de/call-demuxlet/shard-0/demuxlet-0-rc.txt, command failed: CommandException: No URLs matched: /mnt/local-disk/demuxlet-0-rc.txt\nCommandException: 1 file/object could not be transferred.\n); /mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.best -> gs://fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/5e6085a5-f972-453a-9812-83a9b6c8ad9f/DemuxletScatter/ddb5a1a8-f0da-403d-bfb5-e01aebe132de/call-demuxlet/shard-0/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.best (cp failed: gsutil -q -m cp -L /var/log/google-genomics/out.log /mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.best gs://fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/5e6085a5-f972-453a-9812-83a9b6c8ad9f/DemuxletScatter/ddb5a1a8-f0da-403d-bfb5-e01aebe132de/call-demuxlet/shard-0/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.best, command failed: CommandException: No URLs matched: /mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.best\nCommandException: 1 file/object could not be transferred.\n); /mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.single -> gs://fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/5e6085a5-f972-453a-9812-83a9b6c8ad9f/DemuxletScatter/ddb5a1a8-f0da-403d-bfb5-e01aebe132de/call-demuxlet/shard-0/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.single (cp failed: gsutil -q -m cp -L /var/log/google-genomics/out.log /mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.single gs://fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/5e6085a5-f972-453a-9812-83a9b6c8ad9f/DemuxletScatter/ddb5a1a8-f0da-403d-bfb5-e01aebe132de/call-demuxlet/shard-0/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.single, command failed: CommandException: No URLs matched: /mnt/local-disk/fc-6aff27a4-00f0-460b-af6d-db1e0a1f7d40/C11.single\nCommandException: 1 file/object could not be transferred.\n)"

Tagged:

Best Answer

Answers

  • bshifawbshifaw moonMember, Broadie, Moderator admin

    There shouldn't be any permissions issues with accessing the docker image if it is public.
    Do you mind sharing the workspace with [email protected], and telling us the name of the workspace as well as any submission and workflow ids involved?

  • kgosikkgosik Member
    edited July 2018

    The workspace is fasi-regev/fasi-do-mice
    Workflow ID: 66bf7111-73d7-468c-ac7b-91aa7266dff2
    Latetst Submission ID: 46918a88-38a5-47bf-8d7c-f043fa195fa7

    The workspace should be shared with you now. Thank you for looking into this. Please let me know if you have any questions of me.

  • bshifawbshifaw moonMember, Broadie, Moderator admin
    edited August 2018

    The docker was pulled fine. This is indicated in the JES log near the top
    2018/07/30 22:25:15 I: Pulled image "kdgosik/[email protected]:f5c5621f02b7e34244e7ada3b4c8f385e7a453b662610ad5ecada7e735f2124e" successfully.
    As stated in the error log you posted the issue is that the output files of the task are not found. Perhaps the command in the task didn't run successfully or the naming is incorrect. Try editing the wdl to confirm that the command ran fine.

    Post edited by bshifaw on
  • kgosikkgosik Member
    edited August 2018

    I have run this exact WDL script with the input and output all specified the same locally with the same docker image without an issue. It is only when running it on FireCloud the issue arises.

    I edited the WDL script in snapshot 23 to look like below. Where it is just writing an echo statement that is a copy and paste of the output file names and I receive the same error (Workflow ID: cc341e20-cd9a-423f-ac5a-1b2449b04dec, Submission ID: eaa9111f-bb2f-45b6-bc94-3ab4849c0bb9).

    workflow DemuxletScatter {
    
      File inputSamplesFile
      Array[Array[String]] inputSamples = read_tsv(inputSamplesFile)
      String metricField
      String memoryUse = "8GB"
      String diskSpace = "32"
      Int preemptible = 2
    
      scatter (sample in inputSamples) {
        call demuxlet {
    
          input:
            bamFile=sample[0],
            vcfFile=sample[1],
            sampleListFile=sample[2],
            metricField=metricField,
            outputPrefix=sample[3],
            memoryUse=memoryUse,
            diskSpace=diskSpace,
            preemptible=preemptible
          }
        }
    }
    
    task demuxlet {
    
          File bamFile
          File vcfFile
          File sampleListFile
          String metricField
          String outputPrefix
          String memoryUse
          String diskSpace
          Int preemptible
    
      command {
    
        set -e
    
        echo best > "${outputPrefix}.best";
        echo single > "${outputPrefix}.single";
        echo single2 > "${outputPrefix}.sing2";
    
        }
    
       output {
    
        File BestFile = "${outputPrefix}.best"
        File Single1File = "${outputPrefix}.single"
        File Single2File = "${outputPrefix}.sing2"
    
        }
    
      runtime {
    
        docker: "kdgosik/demuxlet:ubuntu"
        bootDiskSizeGb: 12
        cpu: "1"
        memory: "${memoryUse}"
        disks: "local-disk ${diskSpace} HDD"
        preemptible: "${preemptible}"
      }
    }
    
    Post edited by KateN on
  • Great, thank you! That worked

Sign In or Register to comment.