Hi all, I am trying to run a sub-flow on a differe...
# prefect-community
Hi all, I am trying to run a sub-flow on a different docker image than my main flow and pass data back from my sub-flow to my main flow using
prefect 2.0b7-python3.8
. I'm trying to orchestrate the images through
as the flow_runners. Currently what happens is the sub flow runs on the same image as the main flow unless the sub flow is deployed independently. Would greatly appreciate any ideas or insights into current feasibility!
passing data between containers doesn't work natively since those run in entirely different processes, you would need to either use subflows instead or persist the data manually yourself + leverage the orchestrator pattern https://discourse.prefect.io/t/how-to-create-a-flow-run-from-deployment-orchestrator-pattern/803
Hi @Anna Geller, thanks for the response! I think I understand how to do it by "persist the data manually yourself + leverage the orchestrator pattern", but I don't quite understand the "use subflows instead". I thought my current approach was trying to use subflows but maybe I misunderstood.
I think Anna means you need to start a new flow run from a deployment like this
👍 1
subflows pattern means running your flows directly from the main/parent flow orchestrator pattern means triggering your child flow runs via an API call which can be triggered e.g. using
Copy code
👍 1