Hey, i finally deployed prefect on a k8's cluster on aws thanks to your advices ! I was wondering if i could use the

local dask executor

in this cluster or if there is an aditionnal configuration to use dask as an executor in a k8's cluster ?

Hi,

LocalDaskExecutor

is viable and perfectly fine. That will launch a local dask instance within the kubernetes pod your agent launches, and run your flow there. It's a great starting point and sufficient for most use cases. If you want to scale out to multiple pods/machines, You can either: • Use

DaskExecutor

with the url of an already existing dask cluster. • Use something like dask_kubernetes as your

cluster_class

. That will launch pods, run dask on them and clean up when your flow finishes. (I think, never used distributed dask for prefect 😅)

wow ok nice thanks for the infos 🙂