Jackson Nguyen
05/12/2026, 7:07 PMMarvin
05/12/2026, 7:07 PMMarvin
05/12/2026, 7:09 PM1 around the code that touches that hardware.
That lets both workers keep running normally, while ensuring only one flow/task at a time can enter the critical section.
Recommended approach
1. Create a global concurrency limit:
bash
prefect gcl create shared-hardware --limit 1
2. Wrap the hardware access in a concurrency context:
python
from prefect import flow, task
from prefect.concurrency.sync import concurrency
@task
def use_device():
with concurrency("shared-hardware", occupy=1, strict=True):
# exclusive access to the shared device here
print("Using hardware")
@flow
def my_flow():
use_device()
With that in place, if both workers try to run use_device() at the same time, one will wait until the other releases the slot.
Why this is usually better than limiting the workers themselves
- It protects just the shared resource, not the whole worker
- Both workers can still do unrelated work in parallel
- It works across processes/machines because coordination happens through Prefect
Other options
- Work queue concurrency limit: if you want only one flow run from a queue at a time
- Work pool concurrency limit: if you want to cap the whole pool
- Worker --limit: caps how many runs a worker executes, but doesn’t specifically lock the resource
So if the problem is “two workers share one GPU/device/file lock/USB instrument,” the concurrency limit around the task is the cleanest solution.
Useful docs:
- Global concurrency limits
- Tag-based concurrency limits
If you want, I can also show:
- a tag-based version
- how to do this for an entire deployment/queue instead of a single task
- how to handle timeouts/failures so the lock is always released