<@ULVA73B9P>: can you explain to me the design de...
# ask-marvin
w
@Marvin: can you explain to me the design decision why concurrency limit can only be leased for up to 86400 seconds?
m
thought for 272.1 seconds
Great question. Short answer: it’s a safety cap. In Prefect 3.x, concurrency slots are held via leases with a time-to-live. The server enforces a maximum lease_duration of 86,400 seconds (24 hours) per lease/renewal. The goal is to prevent a client from accidentally (or due to a crash/network partition) holding a slot indefinitely. By capping an individual lease’s TTL, the system guarantees eventual, automatic release of orphaned slots and encourages a heartbeat/renewal pattern. A few important clarifications: - It’s not a hard cap on total hold time: you can renew leases repeatedly to keep slots for days or weeks. The 24h limit applies to each individual acquisition/renewal call. - The higher-level
prefect.concurrency
context managers handle acquisition and renewal for you; you usually won’t need to manage renewals manually unless you’re calling the REST API yourself. - The exact “why 24 hours (and not, say, 12 or 48)” is not explicitly documented in a public design note. It’s a pragmatic upper bound that covers long-running workloads while bounding risk from orphaned leases. References - Global concurrency limits guide: How to: Global concurrency limits - API reference (Python server endpoints): Concurrency limits v2 API (Python) - REST API: renew lease: Renew concurrency lease - Leasing utilities overview: Leasing utilities If your use case requires holding a slot longer than 24 hours without any renewals (e.g., an offline process you can’t heartbeat), let me know more about the scenario. We can suggest patterns to fit that workflow or log a feature request to make the max configurable.
w
@Marvin: Ok then, how do I renew the concurrency lease?
m
thought for 162.4 seconds
Great question. You generally don’t need to do anything if you’re using the
prefect.concurrency.concurrency()
or
rate_limit()
context managers—Prefect renews the lease for you while the context is active. If you’re managing leases yourself (e.g., you called “increment with lease” directly), renew by calling the “Renew concurrency lease” endpoint with your
lease_id
and a new
lease_duration
(<= 86,400 seconds). Do this before the lease expires. Example with curl:
Copy code
curl -X POST "$PREFECT_API_URL/api/v2/concurrency_limits/renew-lease" \
  -H "Authorization: Bearer $PREFECT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "lease_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "lease_duration": 300
  }'
When finished with the work, release the slots using the same `lease_id`:
Copy code
curl -X POST "$PREFECT_API_URL/api/v2/concurrency_limits/decrement-with-lease" \
  -H "Authorization: Bearer $PREFECT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "lease_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a" }'
Docs for the endpoints and payloads: - Bulk increment with lease: bulk-increment-active-slots-with-lease - Renew lease: renew-concurrency-lease - Decrement with lease: bulk-decrement-active-slots-with-lease If you want, I can share a small heartbeat loop snippet that periodically renews the lease in a background task.
w
@Marvin: Wow this is bad, like REALLY deal-breaker bad. What is a process of submitting a change request?
m
thought for 163.9 seconds
I hear you—and I’m sorry this is blocking you. We’d love to capture your requirements so we can evaluate a change. Best path to request this change - Open a GitHub issue in the Prefect repo: prefecthq/prefect/issues - Choose “Feature request” and include: - Problem statement: why the 24h max lease is a blocker for your workload - Desired behavior: e.g., a configurable max, a higher default, or “no cap” - Context: are you using the REST API directly or the context manager; how long you need to hold; renewal feasibility; failure modes - Impact: affected scale (number of workflows/users), business impact, any compliance/SLA constraints - Workarounds tried: renewals/heartbeats and why they don’t work Optional but helpful - Repro or pseudo-code of how you acquire/renew/release leases today - Metrics showing how often 24h is exceeded - Any constraints that prevent periodic renewal (e.g., offline air-gapped jobs) If you’d like, I can draft the issue text here based on your use case and post it for you, or you can paste details and I’ll help refine them.