Hi All The prefect tasks which are in resume state are not g

Question

Hi All The prefect tasks which are in resume state are not going through they remain in resume state This is affecting our production flow runs as well Can someone please check if there s any issue with prefect cloud cc < Kevin Kho> < Christina Lopez> < Anna Geller>

Kevin Kho · Answer

Hey < Saurabh Indoria> there is nothing wrong with Prefect Cloud since you posted Is this issue still persisting

Saurabh Indoria · Answer

Yes this is still persisting

Saurabh Indoria · Answer

I can send you the flow run ID if that helps

Saurabh Indoria · Answer

All our production flows are stuck

Kevin Kho · Answer

I could look yep Are your agents healthy

Kevin Kho · Answer

And was this working before and then stopped working

Saurabh Indoria · Answer

Yes we have been using the same flows since months now Below are few examples

Saurabh Indoria · Answer

Agents are healthy as well all tasks run well even the newly created ones just the resume flow functionality is not working

Saurabh Indoria · Answer

This has happened for the second time now couple of months back we saw the same issue which automatically resolved in 2 days without any official release from Prefect And now we are seeing this again and is a bit scary for us

Kevin Kho · Answer

Looking into it

Saurabh Indoria · Answer

Thanks Appreciate any help on this

Kevin Kho · Answer

There is another report of this so I think it s something we need to look at The team is already looking

Saurabh Indoria · Answer

Okay please keep me posted on it thanks

Christina Lopez · Answer

Thanks for tagging me < Saurabh Indoria> so I can be informed

Kevin Kho · Answer

Hey < Saurabh Indoria> we found an API query that was timing out in Cloud for the service that handles resuming these tasks We made edits to it and deployed it and the new deployment just went through but it is still timing out so this is not resolved for now and we are still working on a fix Sorry about that

Saurabh Indoria · Answer

Okay Thanks for the update please keep me posted on further developments

Saurabh Indoria · Answer

< Yash Joshi> CC

Kevin Kho · Answer

This got fixed literally today We rolled out two changes on the backend so stabilize this service so you should not have problems going forward New flows will already work The ones stuck in pending will resolve as the service works through the backlog

Christina Lopez · Answer

< Saurabh Indoria> < Yash Joshi> ^^^

Saurabh Indoria · Answer

Okay great Thanks < Kevin Kho>

Kevin Kho · Answer

Sorry about that

Kevin Kho · Answer

I just checked my flows that were stuck and they are still pending Will talk to the team about that tom but the new ones were definitely working when I tested

Saurabh Indoria · Answer

Sure thanks We also did a hotfix in our production system to move away from pause resume feature until it is stable enough

Saurabh Indoria · Answer

Looks like even new flows get stuck about 15 of the times on Resume state < Kevin Kho> < Christina Lopez> Please get this up on priority if possible

Kevin Kho · Answer

Hi < Saurabh Indoria> sorry about that will check with the team again

Kevin Kho · Answer

Could you give us a new flow run id

Christina Lopez · Answer

< Saurabh Indoria> ^

Saurabh Indoria · Answer

Looks like most of them eventually went through however there certainly is a delay for few flows

Saurabh Indoria · Answer

I couldn t find flow run IDs which are stuck now but once we start testing again on Monday I ll share the flow runs if they get stuck

Kevin Kho · Answer

Yes please