<@ULVA73B9P> Can you help me explain about the wo...
# ask-marvin
t
@Marvin Can you help me explain about the workers? Can 2 workers listen to 1 work pool, 1 work queue without having the race conditions?
m
thought for 618.8 seconds
Short answer: Yes. In Prefect 3.x you can run multiple workers against the same work pool and even the same work queue without double-executing runs. How Prefect avoids races: - When workers poll for work, the server selects scheduled flow runs using database row-locking (Postgres uses FOR UPDATE SKIP LOCKED). This prevents workers from contending on the exact same rows during that poll. - After a worker receives runs, it immediately proposes the run to PENDING. That state change is performed under another database lock. If two workers ever fetched the same run, only the first to set PENDING will succeed; the other will be rejected. Net effect: no run is executed twice. Concurrency controls you can use: - Work pool-wide concurrency limit:
Copy code
prefect work-pool set-concurrency-limit <pool-name> <limit>
- Per work queue concurrency limit:
Copy code
prefect work-queue set-concurrency-limit <queue-name> <limit> --pool <pool-name>
Starting two workers on the same pool/queue (safe to do):
Copy code
# Terminal 1
prefect worker start --pool my-pool --work-queue my-queue

# Terminal 2
prefect worker start --pool my-pool --work-queue my-queue
Notes and best practices: - Multiple workers on the same queue is a common, supported way to scale horizontally. - Prefect adds jitter to polling to avoid “thundering herd” effects. - If a worker crashes after claiming a run (e.g., after moving it to PENDING), there’s no lease/TTL; the run may remain pending and may require intervention or resubmission. Docs for more detail: - Work pools - Workers If you can share your worker type (e.g., Docker, Kubernetes) and desired concurrency, I can suggest concrete settings.