Hi all, is it possible to triage deployments to a work pool dynamically, say, if the first work pool is busy, etc. Either round robin style or anything else? Thanks
@Prefect Bump - any help would be appreciated. Thanks!
j
Jamie Zieziula
08/31/2023, 5:12 PM
Hi @Charles Leung - I don’t think you can specify more than 1 work pool on a deployment
c
Charles Leung
08/31/2023, 5:26 PM
Hi @Jamie Zieziula ah i see. My concern was wanted to, for example, triage jobs to execute on workers dynamically within the work pool. For example, if worker 1 is busy, execute on worker 2. etc
if using one work pool to programthis, is that possible instead?
j
Jamie Zieziula
08/31/2023, 5:29 PM
you can have multiple workers pooling a single work pool! so if one is unavailable, another should pick up and schedule the work
c
Charles Leung
08/31/2023, 5:32 PM
ah makes sense thank you for the quick response @Jamie Zieziula! ONe last question, is there a concurrency limit where we will see performance hits when we run lots of deployments? Im on the server version of Prefect. Can I mitigate this by opening up more work pools?
j
Jamie Zieziula
08/31/2023, 5:42 PM
it would depend on the volume of flows but as long as your infrastructure can scale to handle the flows then you shouldn’t have an issue with the worker. you can always scale the worker vertically if you’re hitting memory constraints