Is it possible to run a prefect subflow in a different worker pool? We have prefect flows all setup to run in k8s pool, for each deployment run, the entire execution happens in a k8s job created by the k8s pool. However, in a special case we would like a subflow to un in a different docker container pool, as we have setup that docker container pool in a specific VM with some necessary information. We thought of making these subflows deployed on docker pool, and trigger it via prefect API, but we are wondering if there are more convenient methods.

Hey @schwannden kuo - you can use the

run_deployment

convenience for this. Check out https://docs.prefect.io/latest/api-ref/prefect/deployments/deployments/?h=run_deployment#prefect.deployments.deployments.run_deployment

This seems like what we need! But in run_deployment, I only see work queue name as parameter, but did not see where to specify work pool name? Since we run the main flow in k8s worker pool, and subflow in docker work pool, is this still the way to go?

If you’ve specified the other work pool on the deployment you’re kicking off, it will send it to the appropriate (docker) work pool. The runtime image question is a good one and im not quite sure. Let me forward this to an internal group and see if i get any thoughts.

thx for the explanation, I think I understand it now, it we create deployment with different image on the same flow, and give it different deployment name, and triigers cooresponding deployment, it should do the trick. I'll experiment on it right away

yep, thats the gist.