Hello, we are testing Prefect to coordinate ML pipelines. We are really happy with the experience so far 🙂. One area that we're still trying to understand fully is resource specification. We are exploring how to run tasks that have heterogeneous requirements (packages and compute) using prefect. Our understanding is that we specify resources/run config at a flow level, and that we could isolate resources/run configs by using a flow of flows. Is this correct, or is there a more straightforward way to specify this at a task level? For reference, we are mostly looking at utilizing kubernetes at this point.

thats correct, config like that is at the flow level

Thanks Zach! Is there an example of running flows with different compute/images?

not sure if there is anything in the docs - at simplebet we use k8s jobs so each flow just has a seperate docker image as needed

@Jason White you can attach a different image to each flow - this is what run config is for in Prefect 1.0 if you're asking about flow-of-flows with different image per child flow, check this page