<@ULVA73B9P> Is retry_jitter_factor only supported...
# ask-marvin
j
@Marvin Is retry_jitter_factor only supported at a task level and not the flow leve?
m
thought for 114.4 seconds
Short answer: yes—retry_jitter_factor is only supported at the task level in Prefect 3.x, not at the flow level. - Flow decorator supports:
retries
,
retry_delay_seconds
(no jitter option) - API ref: prefect.flows API - Task decorator supports:
retries
,
retry_delay_seconds
(can be a number, list, or callable),
retry_jitter_factor
, and
retry_condition_fn
- API ref: prefect.tasks API Docs/how-to on retries: Retries how-to Source for reference: - Flow decorator signature: prefect/flows.py - Task decorator signature: prefect/tasks.py
j
@Marvin Okay. With flow run retries, do these not happen on an crash when submitting to ECS push pool? They seem to only happen on app failures which is not what I'm trying to solve.
m
thought for 174.6 seconds
You’re observing the intended behavior. In Prefect 3.x, flow-level retries only trigger for Failed states, not for Crashed states. With ECS push work pools, infrastructure issues (container OOM, task killed/evicted, SIGTERM, etc.) surface as Crashed, so the flow’s `retries`/`retry_delay_seconds` won’t run. Why: - Failed = application error inside your flow code → flow retries apply - Crashed = infrastructure/interruption → terminal state that does not enter the retry policy Docs: - States: Prefect states - Retries: Retries how-to - ECS guide: ECS integration guide Options to handle crashes: - Prefect Cloud: Create an Automation that triggers on “flow run enters Crashed” and runs your deployment again. This gives you “retry on crash” semantics at the orchestration layer. See Automations. - Infrastructure-level retries: ECS one-off tasks launched via RunTask don’t have built-in retry; if you need infra restarts, consider: - Running via an ECS Service (which can restart tasks), or - Using AWS Batch with a retry strategy and Prefect’s Batch work pool, if job retries are critical. - Reduce crashes: increase CPU/memory, avoid spot interruptions, and ensure graceful signal handling. If your process can catch SIGTERM and exit through normal exception paths, Prefect is more likely to classify as Failed, allowing flow retries to take effect. - Task-level resilience: Keep transient failure handling at the task level (
@task(retries=..., retry_delay_seconds=...)
) to minimize app-level failures, though this won’t help if the entire container dies. If you can share what’s causing the crashes (OOM, SIGTERM during scale-in, spot interruptions, etc.) and whether you’re on Prefect Cloud or OSS, I can recommend a concrete setup (e.g., a specific Automation or an alternative worker/pool).