Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

image.png

Hi everyone,
I want to ask about flow concurrency:
I’ve set flow_concurrency for label `agent:k8s-prod` to 100
But I’ve seen the maximum of concurrent run is only 22-24, the other runs is scheduled
I have 1 kubernetes agent deployment on my k8s cluster

How to make the flow run concurrency more than 24? Anything I miss?
Thanks in advance

It doesn’t even running more scheduled flow run when half done

Like about 40~ started to run in the same time. Maybe any logic behind this, how is the behavior of scheduled runs?

Hey <@ULCV623AT>, I don't know this. I'll ask a team member and get back to you

Hi <@ULCV623AT> - how is your k8s cluster configured? It sounds like you're running into resource constraints somewhere, perhaps your cluster is starved?

I use GKE standard with n1-standard-4 6 node with default configurations, i’m not sure either about the constraint but i just left it with default while creating the cluster. And I test those above while my cluster is empty, no other jobs