Hey folks, we ran into some issues where task concurrency limits were being gobbled up by “ghost” tasks that were still marked running for Flows that were cancelled. We are able to retrieve those with the following GraphQL query:

{
  task_run(
    where: {_and: [{state: {_eq: "Running"}}, {flow_run: {state: {_neq: "Running"}}}]}
  ) {
    id
    start_time
  }
}

Maybe unrelated but for tasks which have a lot of runs, the dashboard seems to hit some kind of query limit and crashes when you try to see the list of Task Runs. The stuck ghost tasks don’t always have a lot of task runs related to them.

@Jacob Blanco I think you may be running into the same issue that was covered in this thread, correct? I will check what’s the current state of this

Yeah certainly seems like it.

@Jacob Blanco just to follow up: we do have an open issue for it internally, so it’s definitely on our radar.