Hi we've recently started to experience problems with flows that goes into AwaitingRetry state. We see on the worker side the following error. "Server sent an abort signal: Cannot transition a flow run with in-process retries to PENDING state" After that the flow is stuck. Has anyone experienced that? @Marvin maybe you can suggest something

The issue you're experiencing with flows getting stuck in the "AwaitingRetry" state due to the error "Cannot transition a flow run with in-process retries to PENDING state" seems to be an uncommon one, as I couldn't find specific references in Prefect's GitHub issues. However, I can point you toward some helpful resources that might give insight into how flow states and transitions work in Prefect, particularly if there have been any recent updates or migration issues between Prefect versions. 1. Prefect 3.x Documentation: I suggest looking at the State Management guide, which discusses how Prefect handles state transitions. Understanding these mechanics might provide some insight into what could cause such an error. 2. Prefect 2.x Documentation: Similarly, if you're using an older version, the State Change Hooks documentation might offer useful information. From the documentation, it appears there could be complexities related to the final state determination of flows, especially when dealing with exceptions and retries. It would be worth checking if any recent changes in your Prefect environment or updates have affected flow behavior according to these principles. If the issue persists, consider reaching out on Prefect's Slack community where you might find more specific insights from other users who have encountered similar issues. If you haven't done so already, updating to the latest stable version of Prefect might resolve the problem, as they periodically patch such issues. And, in the immortal words of Marvin the Paranoid Android, "Here I am, brain the size of a planet, and they ask me to fix flow states!"