<@ULVA73B9P> is there a way to add a cleanup time ...
# ask-marvin
u
@Marvin is there a way to add a cleanup time to prefect jobs and pods once completed in prefect v3?
m
thought for 260.1 seconds
Short answer: Yes—if you’re running on Kubernetes. You can set a TTL so Kubernetes deletes finished Jobs (and their Pods) after X seconds. In Prefect 3.x’s Kubernetes work pools, this is exposed as
finished_job_ttl
which maps to the Job’s
spec.ttlSecondsAfterFinished
. Clarifying question: - Are you using a Kubernetes work pool/worker? If so, the steps below will apply. If you’re using Docker or Process workers, see notes at the end. How to set cleanup TTL for Kubernetes Jobs/Pods - At the pool level (default for all deployments in the pool): 1) Inspect the default template:
Copy code
prefect work-pool get-default-base-job-template --type kubernetes --file k8s-template.json
You’ll see:
Copy code
...
     "spec": {
       "backoffLimit": 0,
       "template": { ... },
       "ttlSecondsAfterFinished": "{{ finished_job_ttl }}"
     }
     ...
2) Set a default TTL by editing pool variables in the UI (Work Pools -> your pool -> Variables) and set
finished_job_ttl
(e.g., 3600 for 1 hour), or bake it into a custom base job template and update the pool:
Copy code
prefect work-pool update "my-k8s-pool" --base-job-template ./k8s-template.json
- Per-deployment override: When deploying, pass the TTL in `job_variables`:
Copy code
from prefect.flows import Flow

  Flow.from_source(
      source="<https://github.com/owner/repo>",
      entrypoint="flows/my_flow.py:my_flow",
  ).deploy(
      name="my-deployment",
      work_pool_name="my-k8s-pool",
      job_variables={"finished_job_ttl": 3600},  # 1 hour
  )
Notes and gotchas - This relies on Kubernetes’ TTL controller. On most modern clusters it’s enabled by default; if TTL is ignored, check with your cluster admin that the TTL-after-finished controller is enabled. - The TTL deletes the Job, which cascades to its owned Pods, cleaning them up after completion. - Don’t confuse this with
job_watch_timeout_seconds
or `pod_watch_timeout_seconds`—those are worker-side timeouts for watching resources, not cleanup. What about non-Kubernetes workers? - Docker work pool: you can auto-remove containers on exit by setting
auto_remove: true
(pool variables or deployment
job_variables
). This cleans up containers but not images/volumes. - Process work pool: nothing to clean—processes exit and the OS reclaims resources. Docs and references - Prefect 3 work pools and templates: Manage work pools and Kubernetes work pools - Kubernetes TTL for finished Jobs: TTL mechanism for finished Jobs If you confirm you’re on Kubernetes, I can share a ready-to-use JSON snippet for your pool template with a sensible default TTL.