Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi everyone, I have my Prefect server running on Azure VM and 1 Prefect Kubernetes agent on AKS.

I scheduled 3 flows at the same time but the problem is that they were submitted by the server one after another. I expected all the flows to run in parallel but that didn't happen. Do I need to increase the number of agents or is it something in the prefect server config that I need to fix.

Hi <@U0208R36B0F>, this should run in parallel. Let me check with the team.

This sounds like k8s resource limits where the cluster doesn’t have enough resources for multiple jobs.

<@U01QEJ9PP53>  But the UI shows that new flow isn't even submitted until the previous flow finishes. I thought it had more to do with the server rather than the k8s cluster. My flows are not computational intensive, they're just simple additions and subtractions.

I see. What are your versions for Prefect Server, the flow, and the agent? The flow run is marked as submitted only when the infrastructure is created. If it’s unable to create the infrastructure, it will remain in a scheduled state. Could you run the agent with debug logs and share what you find if it picks up the concurrent flows?