https://prefect.io logo
Title
f

Florian Guily

05/25/2022, 1:14 PM
Hey, i finally deployed prefect on a k8's cluster on aws thanks to your advices ! I was wondering if i could use the
local dask executor
in this cluster or if there is an aditionnal configuration to use dask as an executor in a k8's cluster ?
e

emre

05/25/2022, 1:55 PM
Hi,
LocalDaskExecutor
is viable and perfectly fine. That will launch a local dask instance within the kubernetes pod your agent launches, and run your flow there. It's a great starting point and sufficient for most use cases. If you want to scale out to multiple pods/machines, You can either: ā€¢ Use
DaskExecutor
with the url of an already existing dask cluster. ā€¢ Use something like dask_kubernetes as your
cluster_class
. That will launch pods, run dask on them and clean up when your flow finishes. (I think, never used distributed dask for prefect šŸ˜…)
šŸ” 2
f

Florian Guily

05/25/2022, 1:57 PM
wow ok nice thanks for the infos šŸ™‚