NoSuchFileException during running JointGenotyping locally

Dear GATK developers,

I have a consistent problem when running the joint-discovery part of your gatk4-germline-snps-indels workflow.
Independent of the file(path) I get a NoSuchFileException which seems to be caused by running it locally and not on the cloud.

Please see my Github issue for details: https://github.com/gatk-workflows/gatk4-germline-snps-indels/issues/13

Please let me know if you need any further information in order to reproduce this.

Kind regards
Tim

Answers

  • bshifawbshifaw moonMember, Broadie, Moderator admin
    edited March 2018

    Hey again TimJeske
    I spoke with a colleague and it looks like the joint_discovery wdl was indeed designed to stream the files listed in the sample map from the cloud. You do have 2 options in performing the JointGenotyping wdl.
    (1) You can use the FireCloud version (joint-discovery-gatk4-fc.wdl) , which accepts input files not as a sample map but as an array which docker will be able mount. The tools in the docker will then be able to perform their task on the mounted files as usual.
    Be sure the dbsnp_vcf is set as a File variable instead of a String. link
    (2) Upload your data to GCS and run the wdl as is.

    Edit: Option 1 requires a few more edits to the wdl, edits will be posted asap.

    Post edited by bshifaw on
  • TimJeskeTimJeske Member

    Hey bshifaw,

    thank you for digging deeper into this issue.
    Due to German data protection regulations, we are unfortunately not allowed to upload our data to any external server, although the cloud computing would make perfect sense of course.

    As I found several differences between the 'normal' and the fc wdl, I would then wait until you provide the edited wdl.
    Can you estimate when you will be able to post it?

    Kind regards,
    Tim

  • bshifawbshifaw moonMember, Broadie, Moderator admin

    The changes to the FC wdl include

    • adding dbsnp_vcf_index = dbsnp_vcf_index to the GenotypeGVCFs call inputs
    • Replacing line 438
      String dbsnp_vcf
      with
      File dbsnp_vcf
      File dbsnp_vcf_index
  • TimJeskeTimJeske Member

    Hey bshifaw,

    thanks for your hints, it did work now!

    I just had to add a few additional parameters to the inputs.json including input_gvcfs, input_gvcf_indices, sample_names and preemptible.
    It's a bit inconvenient to add a lot of duplicate information but I wasn't sure how much I would have to change in the wdl to run it just with the sample_name_map.
    Would it be possible to post a wdl with the changes above that requires just the sample_name_map?
    That would be great!

    Kind regards
    Tim

  • bshifawbshifaw moonMember, Broadie, Moderator admin

    The suggestions above was just for a quick fix, we're currently working on updating that repo.

  • bshifawbshifaw moonMember, Broadie, Moderator admin

    The repo has been updated to include a gcloud and local version for the joint-discovery wdl.

Sign In or Register to comment.