Hey there! Is it possible to run prefect tasks in multiple pods? I have a flow with multiple tasks (fetch_data -> pre-processing -> train_model -> register -> deploy), which I'd like to run in different k8s nodes configuration e.g: pre-processing task in a node with more memory; train_model in a node with GPU.

Hey @Pedro Martins, this would require splitting them up into multiple Flows because you need to change the underlying executor for each of this. You would register these into their own separate flows and have a “main” flow that calls them with the

create_flow_run

task. For each of these separate flows, you would specify the specs when defining the Executor and RunConfig

Or each task in one flow can spawn a Kubernetes job itself, we have a built-in task

RunNamespacedJob

for this

Thank you guys! I think the best solution would be what @Zanie suggested so I can keep a single flow which is easy to maintain. With regards to

RunNamespacedJob

@Zanie, does prefect track the progress and the logs of these jobs in the dashboard?

Yeah it should track status and logs

Cool. I'm going to try.

What you could look at well if you are running with dask. You can specify label on the task to select some special dask worker. I