Holiday Notice:
The Frontline Support team will be slow to respond December 17-18 due to an institute-wide retreat and offline December 22- January 1, while the institute is closed. Thank you for your patience during these next few weeks. Happy Holidays!

python binary missing on 'ubuntu:latest' docker image?

Hello again, now that I'm able to get the simple hello world sample running I've tried going to the next level and tried getting a simple scatter-gather workflow to run. However, I run into this error:

/cromwell_root/exec.sh: line 17: python: command not found

when trying to run this task:

task prepare {
  command <<<
    python -c "print('one\ntwo\nthree\nfour')"
  >>>
  output {
    Array[String] array = read_lines(stdout())
  }

  runtime {
    docker: "ubuntu:latest"
    cpu: 1
    memory: "512 MB"
  }
}

I would've thought that the docker image would have Python installed; did I miss a step?

I've attached prepare.log and exec.sh.txt.

Best Answers

Answers

  • I've also tried running this with the gcloud tool:

    gcloud \
      alpha genomics pipelines run \
      --pipeline-file wdl_pipeline.yaml \
      --zones us-central1-f \
      --inputs-from-file WDL=workflow.wdl \
      --inputs-from-file WORKFLOW_INPUTS=inputs.json \
      --inputs-from-file WORKFLOW_OPTIONS=inputs.json \
      --inputs WORKSPACE=gs://cromwell-test-bucket/wdl_runner/work \
      --inputs OUTPUTS=gs://cromwell-test-bucket/wdl_runner/output \
      --logging gs://cromwell-test-bucket/wdl_runner/logging
    

    and I get this error:

    Failed to connect to Cromwell (attempt 1): ('Connection aborted.', error(99, 'Cannot assign requested address'))

    Does Cromwell have to already be running in server mode somewhere?

  • Looking at the output of stdout.log, it looks like the Cromwell service started but it seems like the problem may be related to the script not being able to find Python?

    ...
    2018-03-06 18:37:37,028 cromwell-system-akka.dispatchers.engine-dispatcher-7 INFO  - Cromwell service started...
    ...
    2018-03-06 18:39:16,863 cromwell-system-akka.dispatchers.engine-dispatcher-22 ERROR - WorkflowManagerActor Workflow 8da957dc-1a2c-4119-a5ff-8f2aaa7f22e7 failed (during ExecutingWorkflowState): Job clearsafety.prepare:NA:1 exited with return code 127 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details.
    Check the content of stderr for potential additional information: gs://cromwell-test-bucket/wdl_runner/work/clearsafety/8da957dc-1a2c-4119-a5ff-8f2aaa7f22e7/call-prepare/prepare-stderr.log
    ...
    

    Then, in stderr.log:

    2018-03-06 18:37:22,476 sys_util INFO: CROMWELL->/cromwell/cromwell.jar
    2018-03-06 18:37:22,476 sys_util INFO: CROMWELL_CONF->/cromwell/jes_template.conf
    2018-03-06 18:37:22,480 discovery INFO: URL being requested: GET https://www.googleapis.com/discovery/v1/apis/storage/v1/rest
    2018-03-06 18:37:22,516 discovery INFO: URL being requested: GET https://www.googleapis.com/storage/v1/b/cromwell-test-bucket/o?fields=nextPageToken%2Citems%28name%29&prefix=wdl_runner%2Foutput&alt=json&maxResults=2
    2018-03-06 18:37:22,517 transport INFO: Attempting refresh to obtain initial access_token
    2018-03-06 18:37:22,604 cromwell_driver INFO: Started Cromwell
    2018-03-06 18:37:22,605 wdl_runner INFO: starting
    2018-03-06 18:37:27,620 cromwell_driver INFO: Failed to connect to Cromwell (attempt 1): ('Connection aborted.', error(99, 'Cannot assign requested address'))
    2018-03-06 18:37:32,625 cromwell_driver INFO: Failed to connect to Cromwell (attempt 2): ('Connection aborted.', error(99, 'Cannot assign requested address'))
    2018-03-06 18:37:38,285 cromwell_driver INFO: Job submitted to Cromwell. job id: 8da957dc-1a2c-4119-a5ff-8f2aaa7f22e7
    ERROR: Status of job is not Submitted, Running, or Succeeded: Failed
    
  • Accepted Answer

    Sorry for all the noise, but want to update this thread in case anyone else runs in to the same problem I did.

    I went back and created and pushed my own Docker image according to the directions in Step 1 here: https://github.com/openwdl/wdl/tree/master/runners/cromwell_on_google/wdl_runner

    Using this image for the rest of the steps, I was able to run my workflow.

  • RuchiRuchi Member, Broadie, Moderator, Dev admin

    Hey @edpark_clearlabs,

    Would you mind pointing out where you found this example task in our docs? We probably should update the docker to be compatible with the task. Thanks!

  • Hey Ruchi, sorry just seeing this now. It's been long ago enough now that I can't recall how I ended up at that point (sorry!)

Sign In or Register to comment.