Bo Ritter
10/21/2025, 1:14 PMMarvin
10/21/2025, 1:14 PMMarvin
10/21/2025, 1:16 PMfrom prefect import flow, task
@flow(timeout_seconds=3600, retries=1, retry_delay_seconds=60)
def my_flow():
...
- Task-level timeout caps long I/O or external calls and can retry independently:
@task(timeout_seconds=300, retries=2, retry_delay_seconds=[30, 60])
def fetch_data(...):
...
- For fine-grained control around a specific block of code:
from prefect.utilities.timeout import timeout, timeout_async
with timeout(seconds=120):
blocking_call()
# or for async:
# async with timeout_async(seconds=120):
# await blocking_call()
Notes:
- On timeout, the run enters a TimedOut failure state. If you’ve set retries, Prefect will retry automatically.
- Links: Timeout utilities
2) Add an Automation to catch “zombie” runs (no heartbeat)
- Prefect runners emit heartbeats (default every 30s). If a run is “Running” but stops heartbeating, it may be stuck/crashed.
- Create an Automation with a proactive trigger that expects a heartbeat or terminal state within a window (e.g., 90–180s). Actions:
- Cancel the flow run
- Optionally trigger a rerun (Run deployment)
- Links:
- Detect zombie flows
- Event triggers and Automations
Optional infrastructure-level guards
- If you run on Kubernetes or other schedulers, you can set job-level limits (e.g., K8s activeDeadlineSeconds) to hard-kill very long runs. Prefer Prefect timeouts for graceful failure/retries; infra-level kills will look like crashes to Prefect.
When to use which
- Use flow/task timeouts for known upper bounds and to enable clean retries.
- Use the zombie-detection Automation to catch deadlocks, crashes, or environment failures where the process stops heartbeating.
If you share your worker type (Docker, K8s, Process) and how long your flows typically run, I can suggest concrete timeout values and a step-by-step Automation setup.