Hi, when I am running multiple workers polling from same work pool, then does prefect ensure load balancing among workers? Also please tell that what underlying scheduling algorithm is followed inside workers(like round robin or something else)?
✅ 1
Lavish Sachdeva
03/07/2024, 8:56 AM
Hey @Jake Kaplan, please help on this as well.
j
Jake Kaplan
03/07/2024, 2:13 PM
Hi, work is not load balanced among workers, it's first come first serve. Only 1 worker can pick up the same flow run. You're able limit the amount of flow runs a worker can submit/monitor at one time with the
--limit
option.
l
Lavish Sachdeva
03/08/2024, 8:39 AM
Hi @Jake Kaplan, can you please elaborate on how does a worker makes execution of different flows concurrent? Suppose that 2 workers are polling from a single work pool, then how is it decided that whether worker-1 will pick the next flow or worker-2, or is it just randomly assigned?
j
Jake Kaplan
03/08/2024, 2:42 PM
It's whichever worker polls for the work first.
Workers are lightweight and designed to deploy and monitor flows to execution environments where your code is actually going to run.