Hey team While executing flows using subprocess on an agent

Question

Hey team While executing flows using subprocess on an agent worker and ConcurrentTaskRunner what s the max no of concurrent flows I can run From what I understand each sub process can only run one flow at a time which means the event loop of that process is shared only by tasks of that flow So assuming the agent is running on a 8vCPU machine can I only have about 16 flow runs in parallel

Zanie · Answer

The agent will spawn a unique process per flow run yes

Zanie · Answer

You can generally exceed far more than your number of cores in number of flow runs i e you have far more processes running on your machine than you do cores However if your flows are computationally intensive you will encounter slowdowns eventually as context switching adds overhead

Aditya N · Answer

Hey < Zanie> are there any caveats to watch out for if process type worker is deployed in production Coz the docs page says this it typically used for getting started testing purposes Thanks

Zanie · Answer

Nope lots of people use it in production

Zanie · Answer

You may want to set some concurrency limits since the flow runs execute locally instead of being managed by something like Kubernetes