<@ULVA73B9P> in Prefect 2.20.16 what is the purpos...
# ask-marvin
s
@Marvin in Prefect 2.20.16 what is the purpose of the task timeout option on the decorator?
m
thought for 95.6 seconds
In Prefect 2.20.16, the
timeout_seconds
option on
@task
sets a maximum runtime for that task run. Example:
Copy code
python
from prefect import task

@task(timeout_seconds=30)
def my_task():
    ...
That means Prefect will allow
my_task
to run for up to 30 seconds. If it takes longer than that, Prefect times it out and marks the task run as failed with a
TimedOut
state. Purpose - Prevent a task from hanging forever - Fail slow/stuck work quickly - Let retries kick in automatically if configured What happens on timeout - The task execution is interrupted by Prefect - The task run is converted into a failed state named
TimedOut
- If the task has retries configured, the timeout counts like a normal failure and Prefect can retry it Important caveat - The timeout is per attempt, not for the whole task across retries. - So if you set
timeout_seconds=10
and
retries=2
, the task could potentially get up to ~10 seconds on each attempt. So in short: it’s there to cap how long a single task attempt is allowed to execute before Prefect fails it. If you want, I can also explain how this differs from flow-level timeouts or how it behaves with async tasks in 2.20.16.
s
@Marvin if a task hangs for whatever reason and the timeout is set to 24 hours, will the task remain active until it is cancelled manually or the 24 hours is elapsed?
m
thought for 8.9 seconds
Yes — in normal Prefect 2.20.16 behavior, the task will remain running/active until one of these happens: - the task finishes - the task is manually cancelled/stopped - the
timeout_seconds
value is reached So if
timeout_seconds=86400
(24 hours), a hung task can remain active for that full 24-hour window unless something external stops it first. A few caveats: - Prefect’s timeout is not a health check; it does not detect “hung but unhealthy” earlier than the configured timeout. - If the underlying work is blocked in a way that is hard to interrupt, Prefect may mark the task as timed out, but cleanup of the underlying process/thread can depend on the execution environment. - If you need faster detection of hangs, set a shorter
timeout_seconds
and use retries. If you want, I can also explain how this differs depending on whether the task is running: - in-process locally, - on a worker, - in Kubernetes / Docker / subprocess-based infrastructure.
s
@Marvin the task is running in Kubernetes
m
thought for 128.8 seconds
In Kubernetes, the key thing is:
timeout_seconds
on a task does not tell Kubernetes to kill the pod.
It is enforced by Prefect inside the Python process running in the pod. So if your task hangs: - the task run stays active until the timeout elapses - when the timeout elapses, Prefect marks the task as
TimedOut
- the pod itself is not automatically killed just because the task timed out - the pod will keep running until the overall flow process exits, retries continue, or something external stops the pod/job So for your specific question: Yes, if the task is hung and the timeout is 24 hours, it can remain active for that full 24 hours unless you manually intervene first. A useful distinction: - task timeout = limits a single task attempt in Prefect - Kubernetes pod/job lifetime = controlled separately by Kubernetes / job settings / worker behavior If this is a Prefect 2 flow running as a single flow-run pod, then usually: - tasks execute in the flow-run process - a hung task means that that pod stays running - after timeout, the task fails, and then the flow may retry, fail, or continue depending on your code/config Important caveat: if the task is stuck in uninterruptible I/O or some blocking native call, Prefect may mark it timed out at the orchestration level, but the underlying process behavior can still be messy. In practice, that’s another reason to pair task-level timeouts with Kubernetes-level limits. If you want stronger guarantees in Kubernetes, use both: - a reasonable
timeout_seconds
on the task - a Kubernetes/job-level timeout such as
activeDeadlineSeconds
or the equivalent infrastructure setting for your Prefect setup If you want, I can help you map this to your exact Prefect 2 Kubernetes setup: -
KubernetesJob
infrastructure - Kubernetes agent - Helm worker/agent setup - one flow run per pod vs Dask/Ray task pods