When my tasks crash that use concurrency limit slo...
# ask-community
m
When my tasks crash that use concurrency limit slots, they don't seem to clear out of the concurrency limit queue ever. Is there a way to set a timeout on concurrency limit slots? Now none of the preceding tasks can run because the slots are full with tasks they think are still running but they are not.
b
Hey Mike! I have a few clarifying q's for you: • the tasks are entering a
Crashed
state, is that correct? • what version of prefect are you using currently (output of
prefect version
)?
m
Hey Bianca, Thanks for getting back to me. The Flow that creates the tasks is in a crashed state. The tasks them self get stuck in a Running state indefinitely so they never get removed from the task concurrency queue and the next time the flow runs, the task concurrency limit is reached until I delete the concurrency queue and re-create it to clear the stuck running tasks in it. I am using Prefect version 2.10.4
b
Hey Mike, this seems related to this open issue. From what I understand, a fix for this is currently being planned by our engineers.
In the meantime, we do have a method to reset the concurrency limit on a specific tag. It may be beneficial to create an automation which triggers a helper flow that resets the concurrency limit for the affected tag in response to a flow entering a
CRASHED
state
m
OK sounds good Bianca! Thanks for the information. Any line of sight into when that issues fix will be merged? Looks like it's been open since July 2022
g
FYI, I still have plenty of tasks creashing in subflow that do not stop the main flow. The flow is in "running" state why the sub flow is crashed. This plays badly with the concurrency limit...