<@ULVA73B9P> if a flow got stuck in pending (the w...
# ask-marvin
t
@Marvin if a flow got stuck in pending (the worker that submitted it got killed), does it auto recover?
m
thought for 192.5 seconds
Short answer: no, not by itself. If a flow run has already been moved to Pending and the worker dies/killed before the run actually starts, that run can remain in Pending indefinitely. A few important details: - Scheduled runs can be picked up by another worker - But once a worker has claimed/submitted the run and set it to Pending, Prefect treats that as “submission in progress / infra being started” - The actual move from Pending -> Running happens when the flow process starts executing - If the worker dies in that gap, there is not a built-in automatic recovery/timeout for that stuck Pending flow run So in practice: - If the worker died before claiming the run: another worker can still pick it up - If the worker died after setting it to Pending but before execution began: the run may get stranded in Pending - A replacement worker will not usually “adopt” that same pending run automatically What you can do: - Manually cancel or retry/reschedule the stuck run in the UI - Or do it via the API if you want automation around stale pending runs - Restarting workers helps with future scheduled work, but won’t usually unstick the already-pending run Given your setup, this is especially relevant if you’re already handling missing-heartbeat/crash logic — Pending is a separate stuck state you may want to detect explicitly. If you want, I can help you build: - a small automation/script to find Pending runs older than X minutes and reschedule them, or - guidance on how to distinguish “safe to reschedule” from “infra may actually have started.”