Specify output on S3 using AWS Batch backend

I am trying to specify the output of my pipeline on s3 using the AWS Batch backend.
Trying the example workflow in the AWS tutorial runs fine as is.
When I create an options.json file in which I specify the output I get an error.

I run cromwell as such:

java -Dconfig.file=awsbatch/aws.conf -jar cromwell-36.jar run awsbatch/hello.wdl -o options.json -i awsbatch/hello.inputs

and the options.json file is:

{
    "final_workflow_outputs_dir": "s3://<s3_uri>/cromwell/outputs",
    "final_workflow_log_dir": "s3://<s3_uri>/cromwell/wf_logs",
    "final_call_logs_dir": "s3://<s3_uri>/cromwell/call_logs"
}

The error output :

[2019-01-07 19:18:24,38] [info] $a [c4bdf097]: Copying workflow logs from /Users/kmavrommatis/workspace/Workflows/cromwell/cromwell-workflow-logs/workflow.c4bdf097-6f1d-4afe-b99d-38fae9d80f7e.log to s3://<s3_uri>/cromwell/wf_logs/workflow.c4bdf097-6f1d-4afe-b99d-38fae9d80f7e.log
[2019-01-07 19:18:24,84] [error] Key cannot be empty
java.lang.IllegalArgumentException: Key cannot be empty
    at software.amazon.awssdk.core.util.ValidationUtils.assertStringNotEmpty(ValidationUtils.java:111)
    at software.amazon.awssdk.core.runtime.transform.PathMarshallers$GreedyPathMarshaller.marshall(PathMarshallers.java:109)
    at software.amazon.awssdk.services.s3.transform.HeadObjectRequestMarshaller.marshall(HeadObjectRequestMarshaller.java:87)
    at software.amazon.awssdk.services.s3.transform.HeadObjectRequestMarshaller.marshall(HeadObjectRequestMarshaller.java:31)
    at software.amazon.awssdk.core.client.SyncClientHandlerImpl.execute(SyncClientHandlerImpl.java:88)
    at software.amazon.awssdk.core.client.SyncClientHandlerImpl.execute(SyncClientHandlerImpl.java:76)
    at software.amazon.awssdk.core.client.SdkClientHandler.execute(SdkClientHandler.java:45)
    at software.amazon.awssdk.services.s3.DefaultS3Client.headObject(DefaultS3Client.java:1628)
...

Can you please advise?
Thanks in advance for your help

Sign In or Register to comment.