<@ULVA73B9P> in prefect 2.20, is it possible to ha...
# marvin-ai
f
@Marvin in prefect 2.20, is it possible to have multiple workers connected to the same queue? How does load balancing happen in that case?
m
thought for 395.9 seconds
Yes. In Prefect 2.20, multiple workers can poll the same work queue. The distribution is competitive rather than round-robin: - Each worker independently polls for scheduled runs. - A worker attempts to claim a run by transitioning it from
Scheduled
to
Pending
. - The server serializes that transition; if multiple workers see the same run, only one
Pending
transition succeeds. The others abort submission. - Therefore, faster workers or workers with more free capacity may receive more runs—there is no worker-level fairness guarantee. Capacity is controlled at several levels: - Worker `--limit`: maximum simultaneous runs for that individual worker. - Work-queue concurrency limit: maximum active runs from that queue. - Work-pool concurrency limit: maximum across the pool. So running several identical workers against one queue is the normal way to scale horizontally and add redundancy. Sources: Prefect 2.20 work pools and workers, worker polling implementation, server-side state transition