I am using prefect cloud, and for some reason only...
# ask-communityj
Jacob Goldberg
03/01/2024, 8:06 PMI am using prefect cloud, and for some reason only 1 flow will run at a time, i cannot for the life of me figure out why. I have a Global concurrency limit set, but it is set to 3. I also have a work pool concurrency limit set (also set to 3). Yet more than one flow will not run at once. Any ideas?
_edit: I also checked to ensure there were no concurrency limits on the work_queue as well via prefect CLI, and confirmed there were none._
n
Nate
03/01/2024, 8:15 PMdo you have old runs stuck in
PENDINGRUNNINGj
Jacob Goldberg
03/01/2024, 8:17 PMThere are no flows in the pending on running state (except the one that runs at a time)
Jacob Goldberg
03/01/2024, 8:18 PMhold on… actually this may be it. I was looking in the last 2 weeks, but going back 2 months a see a flow stuck in pending
Jacob Goldberg
03/01/2024, 8:19 PMaaaand I found another one that was stuck from earlier in 2023. That was it! thanks 🙂 I feel kind of dumb, but also not sure how those were stuck for so long….
n
Nate
03/01/2024, 8:19 PMno worries!
this has been a common gotcha and we are actively thinking of ways to surface that "your flows are blocked by a full concurrency limit" and automatically deal with it
👍 1
Nate
03/01/2024, 8:20 PMin the meantime if you find yourself in need of a script to go back and find / delete / reschedule old runs ill point you here
https://docs.prefect.io/latest/guides/using-the-client/?h=client#rescheduling-late-flow-runs
🙌 1
s
staticnotdynamic
03/27/2024, 12:16 PMHello Jackob, sorry for hijacking this thread here, but I cam here for help and read your post.
Hey @Nate, I have the same excat problem. All my work queues have concurrency level of 1, and if one flow run is stuck in "pending" state, all next runs are blocked!
staticnotdynamic
03/27/2024, 12:17 PMI'd appreciate any help. This is Prefect V2.