Hi everyone, I have a big flow with hundreds of sm...
# ask-community
j
Hi everyone, I have a big flow with hundreds of smaller tasks that run in parallel. Currently, a flow run is processed in only one k8s pod. Is it possible to process those tasks in multiple pods at the same time?
1
d
Might be a usecase of sub flows.
j
I've tried subflows but they run in the same pod as the parent flow. Is it correct behavior or should Prefect create new k8s jobs/pods for subflows?
d
If you directly call subflows it runs in the same env as the parent. But You can use have the subflow as a seperate deployment and call run_deployments method to have those subflows running as a seperate flow in new pods.
🙌 1
j
Thanks, I'll try it 👍