Im trying to use Prefect with a dask kubernetes cluster (installed with helm stable/dask) - however whenever i install prefect on the dask worker pods it upgrades distributed and then the cluster falls apart - any one got prefect working in this way?

Hey Alistair - that’s interesting; what version of distributed are you working with? If the version falls below

1.26.1

, then I could see why your package manager would upgrade it — if not though, we’ll have to think about possible workarounds here 🧐

i created this issue on dask-kubernetes before i worked out that it was my inclusion of prefect in the "EXTRA_PIP_PACKAGES" section of the config.yaml passed to helm. https://github.com/dask/dask-kubernetes/issues/161

Ah ok very interesting. So you need `1.25.2`; I would need to revisit, but I believe we upgraded our minimum distributed requirement to

1.26.1

to include some distributed features that we needed in deployment.

I dont need 1.25.2, thats just whats bundled on the devdask/dask:1.1.0 container that the chart uses

ok great - what is it?

if i install prefect on the scheduler through the config.yaml used by helm it upgrades distributed on the other pods to 1.28.1

ahhh yes, very nice

im just testing now

great!