We've moved!
This site is now read-only. You can find our new documentation site and support forum for posting questions here.
Be sure to read our welcome blog!

CallCaching Query

Hello, and apologies if this is not the correct place for this query.

Cromwell engine version 46

We are currently looking at a Proof of Concept with Cromwell and AWS batch.
We trying to understand the CallCaching trigger(s), as we require this if the next step in a multi-step workflow breaks.

Currently, we have set up the in-memory version and are not using any form of database.

We have added the following to the configuration file:
call-caching { enabled = true invalidate-bad-cache-results = true }

Question 1.

Can call caching be initiated if there is only the in-memory database, as below is not clear regarding this?
"Cromwell's call cache is maintained in its database. In order for call caching to be used on any previously run jobs, it is best to configure Cromwell to point to a MySQL database instead of the default in-memory database. This way any invocation of Cromwell (either with run or server subcommands) will be able to utilize results from all calls that are in that database."

Secondly, if this can.

Question 2.

Can call caching be initiated if a scatter, wraps a workflow, which then wraps tools.
Or will the entire workflow need to be in one script? (I can provide the full zip of scripts and options etc if required)
And, the options file.
DsTrim - Broken.zip

Question 3.

What exactly triggers callcaching to change from "CallCachingOff" to on, in the following result?
"callCaching": { "effectiveCallCachingMode": "CallCachingOff", "allowResultReuse": false, "hit": false, "result": "Cache Miss" },

If the in-memory is the issue, then please close and we will set-up a UAT correctly.
If not any additional assistance or comments will be most apprecitated.

Best Answer

Answers

  • bhanuGandhambhanuGandham Cambridge MAMember, Administrator, Broadie, Moderator admin

    Hi @dbridges

    I have informed the WDL/Cromwell support team about your question and some one will get back to you shortly.

  • dbridgesdbridges UKMember
    @SChaluvadi thank you for this. This was what I had expected wrt in-memory vs DB and was hoping with reference to the calling at the workflow and subworkflowlevel.

    We have now set up a UAT with mysql database, and call-caching works for the hello-world example, however I am unable to get it to work for a 3 tier scatter at the top level, call a workflow at the second level and individual tasks at the third level.

    Is there something that needs to remain consistent in order for the call caching to cache?

    I can see in the database that is starts but then hits a Cache Miss and switches off for some reason that I am not sure why?
    Any help in this matter would be most appreciated.
Sign In or Register to comment.