Thanks for the details — that message means the concurrency lease your run was holding expired and could not be renewed, so Prefect cancels execution to avoid running without reserved slots. What’s going on - Prefect enforces concurrency limits with renewable “leases.” While your flow/task is running under a limit, a background heartbeat renews the lease with the server before its TTL expires. - If renewals stop (e.g., long event-loop blocking, network hiccups, server restart) and the TTL passes, the server releases the slots. The runtime then raises: “Concurrency lease renewal failed - slots are no longer reserved. Terminating execution to prevent over-allocation.” Common causes we see - Long-running, blocking code that prevents the lease-renewal coroutine from running in time (especially in async flows/tasks doing blocking work). - A flow/task timeout firing first (cancelling the run) and then the lease renewer shuts down — you’ll see “Flow run exceeded timeout …” right before the lease error. - Transient networking issues between worker and API, or API restarts. - Very long jobs running entirely under a concurrency guard or tag-based limit. How to prevent it for long-running work - Don’t hold a concurrency lease longer than needed - Only wrap the resource-critical section in a concurrency guard instead of the entire long-running task/flow. - For tag-based limits, avoid tagging the entire long task if the limited resource is only used briefly. - Increase the lease TTL if you’re using the concurrency context manager - Prefect renews leases for you, but a longer TTL gives more margin during brief pauses. - Example (sync):

from datetime import timedelta
    from prefect.concurrency.sync import concurrency

    def do_work():
        with concurrency("gpu", occupy=1, lease_duration=timedelta(hours=6)):
            long_blocking_call()

- Note: the API caps a single lease_duration at 1 day; renewals continue beyond that if heartbeats succeed. See issue #19218. - Ensure the event loop isn’t blocked - If your flow/task is async, run blocking code in a thread or process so the lease renewer can run:

import anyio

    async def do_work_async():
        await anyio.to_thread.run_sync(long_blocking_call)

- Check and relax timeouts - If you set

timeout_seconds

on your flow or tasks, increase or remove them so they don’t cancel the run before renewal happens. - Stabilize worker ↔️ API connectivity - Ensure your worker can continuously reach the API (Kubernetes network policies, idle timeouts, proxies, etc.). We’ve seen lease renewal failures during brief API outages or pod restarts. - As a last resort, disable or avoid the limit - Remove the concurrency-limited tag or don’t enter a

concurrency(...)

section for that long task (or temporarily raise the limit if it’s safe). References and related threads - Tag-based and global concurrency limits: Tag-based limits and Global limits - Lease renewal API: renew-concurrency-lease - Report with the exact error and stack: issue #19068 - 1-day lease duration cap discussion: issue #19218 A few quick questions to tailor the fix: 1) Are you using tag-based concurrency, the

prefect.concurrency.*.concurrency

context manager, or a deployment-level concurrency limit? 2) Roughly how long do the affected tasks/flows run, and do you have

timeout_seconds

set on the flow or tasks? 3) What Prefect versions are you running (worker and server), and what environment (local, Docker, Kubernetes)?

If you share those, I can suggest the safest concrete change (code snippet or config) for your setup.