Syméon del Marmol
09/18/2025, 5:58 PMMarvin
09/18/2025, 5:58 PMMarvin
09/18/2025, 6:02 PMPREFECT_API_SERVICES_LATE_RUNS_AFTER_SECONDS
), but they are not auto-cancelled unless you add an Automation to do so.
- If a run starts then disappears (no heartbeat), it can be detected as a zombie and marked Crashed, but there’s no “cancel after N minutes in Scheduled/Pending” default.
- Details: Detect zombie flows, States
A bit more color for your background-task usage
- When you cancel a flow run (UI/API), Prefect will propagate cancellation to all running tasks, including background tasks started via .delay()
. Calls to .result_async()
will raise cancellation (e.g., `CancelledError`/`CancelledRun`) so you can handle/cleanup.
- If one background task crashes/fails/cancels, your flow will only fail/cancel if you await its result and let that exception propagate; otherwise you can handle it and continue.
- Pattern with asyncio.gather:
# fan out
futures = [
my_task.with_options(name=f"job-{i}").delay(i)
for i in items
]
try:
results = await asyncio.gather(
*(f.result_async() for f in futures),
return_exceptions=True, # important: collect per-task outcomes
)
for r in results:
# handle per-task outcomes however you prefer
# - Cancelled: treat as skipped/cancelled
# - Exception: handle failure
# - Normal value: success
...
except Exception:
# If you want to let one failure cancel the whole flow, re-raise here.
raise
Common follow-ups you might consider
- If you want “auto-cancel if not started within X minutes”, create an Automation that triggers on Late (or still Scheduled after N minutes) and transitions the run to Cancelled.
- If you want “auto-cancel if a task takes longer than X”, set a timeout_seconds
on the task (via decorator or .with_options(timeout_seconds=...)
) so Prefect cancels it.
Docs for reference
- States: States
- Cancellation: Cancel workflows
- Zombie/crash detection: Detect zombie flows
If you can share whether you’re on Cloud vs OSS and whether you want time-based auto-cancellation, I can suggest a concrete Automation or configuration that matches your fan-out pattern.