@Marvin i have a k8s worker. when i call a sub-flow , does it create an entirely new k8s job?
n
Nate
06/05/2024, 6:22 PM
no, subflows run in process
however if you use
run_deployment
, the resulting flow run will behave like a flow run and will also have whatever infra that deployment has setup
m
Marvin
06/05/2024, 6:23 PM
When you call a sub-flow in Prefect using a Kubernetes worker, it indeed creates a new Kubernetes job for that sub-flow. Prefect spins up new infrastructure resources for each flow and sub-flow run.
For more details, please refer to the following resources:
1. Upgrade guide: agents to workers
2. Infrastructure concepts
If you have any other questions or need further clarification, feel free to ask.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.