https://prefect.io logo
a

Alistair Hey

07/05/2019, 12:28 PM
Im trying to use Prefect with a dask kubernetes cluster (installed with helm stable/dask) - however whenever i install prefect on the dask worker pods it upgrades distributed and then the cluster falls apart - any one got prefect working in this way?
c

Chris White

07/05/2019, 2:54 PM
Hey Alistair - that’s interesting; what version of distributed are you working with? If the version falls below
1.26.1
, then I could see why your package manager would upgrade it — if not though, we’ll have to think about possible workarounds here 🧐
a

Alistair Hey

07/05/2019, 2:57 PM
i created this issue on dask-kubernetes before i worked out that it was my inclusion of prefect in the "EXTRA_PIP_PACKAGES" section of the config.yaml passed to helm. https://github.com/dask/dask-kubernetes/issues/161
stable/dask installs version 1.25.2
c

Chris White

07/05/2019, 3:00 PM
Ah ok very interesting. So you need `1.25.2`; I would need to revisit, but I believe we upgraded our minimum distributed requirement to
1.26.1
to include some distributed features that we needed in deployment.
you might be able to install
prefect
with no-dependencies, and then install the other dependencies that prefect requires manually to temporarily work around this issue
a

Alistair Hey

07/05/2019, 3:02 PM
I dont need 1.25.2, thats just whats bundled on the devdask/dask:1.1.0 container that the chart uses
think iv found a workaround actually
c

Chris White

07/05/2019, 3:03 PM
ok great - what is it?
a

Alistair Hey

07/05/2019, 3:04 PM
if i install prefect on the scheduler through the config.yaml used by helm it upgrades distributed on the other pods to 1.28.1
c

Chris White

07/05/2019, 3:04 PM
ahhh yes, very nice
a

Alistair Hey

07/05/2019, 3:04 PM
im just testing now
yes - that appears to have worked. Thanks
🎉 2
c

Chris White

07/05/2019, 3:05 PM
great!
👍 1
@Marvin archive “Issue running Prefect on dask cluster installed with helm”
7 Views