Unexpected failure in EJEA :

esalinasesalinas BroadMember, Broadie ✭✭✭
edited September 2017 in Ask the FireCloud Team

I actually got this error message for FOUR workflows seemingly at the same time. At present, I have one open in my browser. Other workflows showed a similar message, but I closed their tabs, then tries to re-open them, but upon attempting to view them I got timeout errors. (see below). I am guessing if I close the tab that has the error still visible that I'll get a timeout if I view it:

https://portal.firecloud.org/#workspaces/broad-firecloud-testing/hg38_PoN_Creation_copy/monitor/ee5b45d3-5651-4907-86d4-f656208490c3/af54751e-cb90-45a2-b4a9-83b911a97c13

Submission ID
ee5b45d3-5651-4907-86d4-f656208490c3
Workflows:

Workflows
SUCCESS
Workflow ID:af54751e-cb90-45a2-b4a9-83b911a97c13
Status:
Call Caching:Enabled
Submitted:September 28, 2017, 4:33 PM (22 hours ago)
Started:September 29, 2017, 2:28 PM (8 minutes ago)
Ended:September 29, 2017, 2:28 PM (7 minutes ago)
Inputs:Show
Outputs:None
Workflow Log:workflow.af54751e-cb90-45a2-b4a9-83b911a97c13.log
Workflow Timing:Show
Failures:Hide
message: Unexpected failure in EJEA.
causedBy: 
message: java.util.concurrent.RejectedExecutionException: Task [email protected] rejected from [email protected][Running, pool size = 200, active threads = 200, queued tasks = 2000, completed tasks = 28926]
causedBy: 
message: Task [email protected] rejected from [email protected][Running, pool size = 200, active threads = 200, queued tasks = 2000, completed tasks = 28926]

(the timeout errors are like this : as seen in the network tab with a generic "Server Unavailable" message:)
see also : https://gatkforums.broadinstitute.org/firecloud/discussion/10417/request-timed-out-submission-on-sample-set-entity-with-8891

Server Unavailable
FireCloud service is temporarily unavailable. If this problem persists, check http://status.firecloud.org/ for more information.

{
  "statusCode": 504,
  "source": "FireCloud",
  "timestamp": 1506710342744,
  "causes": [],
  "stackTrace": [],
  "message": "Request Timed Out"
}

The four workflows I think that have this are:
https://portal.firecloud.org/#workspaces/broad-firecloud-testing/hg38_PoN_Creation_copy/monitor/ee5b45d3-5651-4907-86d4-f656208490c3/af54751e-cb90-45a2-b4a9-83b911a97c13
https://portal.firecloud.org/#workspaces/broad-firecloud-testing/hg38_PoN_Creation_copy/monitor/691d14af-1bb5-4bd8-b3cf-76c92b94fda3/ddf5485e-c59f-43f3-a060-0dbdb4e33467
https://portal.firecloud.org/#workspaces/broad-firecloud-testing/hg38_PoN_Creation_copy/monitor/f788f8b4-0f3b-4cc3-a34b-faab57123b65/e6b1c48c-b141-4281-91c6-c3b400b13454
https://portal.firecloud.org/#workspaces/broad-firecloud-testing/hg38_PoN_Creation_copy/monitor/9adb666c-3151-4ed7-9421-21ffcdc12e1f/58e415b0-8f55-4294-8a76-ffcde5e29ea7

this workspace has been previously shared with group_support.

I actually ran the workflow several times anticipating that one or more would fail for some reason, but they all failed at the same time : September 29, 2017, 2:28 PM

Answers

Sign In or Register to comment.