Heads up:
We’re moving the GATK website, docs and forum to a new platform. Read the full story and breakdown of key changes on this blog.

Task hanging without producing JES log/stderr/stdout

mleventhalmleventhal Cambridge, MAMember, Broadie ✭✭

Hello,

I have a task in Firecloud that will start and will hang without producing a JES log, stderr or stdout file. Even stranger, if I do not cache calls, outputs will be produced in the google bucket, but still without the log files and the task still hangs. It is with one particular task in my method configuration, every other task and any other method configuration works fine. It even shares a docker image with a task that works perfectly fine so I suspect that is not the issue. Additionally, the task behaves well locally and does not run into this issue. Let me know if this sounds like anything that has been seen before or if I need to share the method config. Thank you!

Best,
Matt Leventhal

Answers

  • mleventhalmleventhal Cambridge, MAMember, Broadie ✭✭

    For posterity, I solved the issue: it turns out Firecloud does not inform you if a foreign parameter (in my case "bootDiskSize" as opposed to "bootDiskSizeGb") is incorrect. In fact, cromwell was still able to run this wdl locally even with the error, but in Firecloud, this causes the issue described above where no logs are produced and the job remains in "submitted" status limbo.

  • mleventhalmleventhal Cambridge, MAMember, Broadie ✭✭

    This issue is specific to foreign parameters in the Runtime block

  • KateNKateN Cambridge, MAMember, Broadie, Moderator admin

    That is concerning to get an error and not have it reported anywhere. I know locally, Cromwell ignores runtime parameters that it cannot do anything with (such as the preemptible setting) so that you can use the same WDL locally and on FireCloud. However, FireCloud should handle the error better than simply hanging. I will put in a bug report for this, thank you for your feedback.

Sign In or Register to comment.