Hi, does anyone know if the prefect-dask lib shoul...
# prefect-community
s
Hi, does anyone know if the prefect-dask lib should be able to handle the operator.KubeCluster from dask_kubernetes?
1
a
you could open a feature request on the prefect-dask repo, but afaik this is not currently supported you could use DaskTaskRunner with a Dask cluster deployed to K8s with KubeCluster though
s
Yep. Trying to get that running but feels buggy.
Might be that setting it up from a kubernetes job works better but need to test it still though.
🙌 1
s
@Sander just curious how you have ended up proceeding. I have some embarrassingly parallel tasks in a job that would benefit from running as dask tasks on a dask kubernetes cluster. You mentioned that doing so feels "buggy". I'm not looking for deep insight, but really just enough to know if we should pursue dask tasks on kubernetes right now.
s
@Sean Davis still work in progress unfortunately. No success yet.