https://prefect.io logo
Title
m

Michael Smith

05/24/2022, 12:13 PM
Hello, with Prefect 1 on K8s what are the options for partitioning workloads so that we can prioritise "interactively" triggered workflows vs pure batch?
a

Anna Geller

05/24/2022, 12:18 PM
Can you explain the problem you are trying to solve? We don't have any priority queue, but you can apply concurrency limits when you are on Prefect Cloud to queue runs so that you don't overwhelm the agent with too many simultaneous runs (if that's the problem you are trying to optimize for here)
m

Michael Smith

05/24/2022, 12:24 PM
Hi Anna, we're running load tests on our prefect installation and have been making adjustments in terms of cpu allocation, number of replicas and cluster node scaling. Our target is to support 1000 parallel workflows, however its possible that we could run for the medium term with reduced capacity (say 500 concurrent workflows). If we're limited to 500 concurrent workflows then we'd need to be able to prioritise workflows that have been triggered by a direct user action (i.e. the user is waiting for it to complete) whereas background/batch operations could be allowed to trickle through at a slower pace
a

Anna Geller

05/24/2022, 12:36 PM
Are you on Prefect Cloud or on Server?
You could have one more powerful agent and when triggering flow runs manually via a Quick Run in the UI, you could provide a different agent label pointing to that more capable machine
m

Michael Smith

05/24/2022, 1:04 PM
We're on Server
a

Anna Geller

05/24/2022, 1:10 PM
for Server performance, check this detailed discussion