Quick question on the Dask Cluster tutorial: Gett...
# prefect-community
Quick question on the Dask Cluster tutorial: Getting the following error when the flow tries to run
Copy code
distributed.core - INFO - Starting established connection
distributed.core - ERROR - add_client() got an unexpected keyword argument 'versions'
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/core.py", line 412, in handle_comm
    result = handler(comm, **msg)
TypeError: add_client() got an unexpected keyword argument 'versions'
dask-scheduler, version 2.2.0
Prefect, version 0.9.0
. Makes me think a version issue?
Hey @Chris O'Brien - this is caused by a version mismatch between the dask-distributed version that your Flow is using and the version of dask that the cluster is using — where are you spinning this cluster up?
Ah I had observed this earlier. Looks to be a version issue w/ distributed after this recent change https://github.com/dask/distributed/commit/32cb96effe9287fdc42dd73be36989be13abe99d#diff-6a3108098afcddf88b38652ae1e383b5 The
version in the prefecthq/prefect docker image is
but the version of distributed that your flow is using is probably higher. Are you building your flow with a Docker storage?
Ahhhhh, so for my Prefect project it has dask=2.9.2
Copy code
"dask": {
            "version": "==2.9.2"
Thinking I should downgrade this?
yea you could downgrade that to
and it should begin working; you could also build your own custom image to use within the cluster if you use any 2.9.2-specific features
Ooooooof, sometimes Python is a hell of a time. So got the communication working and just working through some exceptions now!
😁 1
Interested in understanding this failure
distributed.client - WARNING - Couldn't gather 37 keys, rescheduling
Hmmm that suggests that your dask cluster is having a networking problem of some kind
is this being run in a tightly configured k8s cluster of some kind?
Heh, nothing of the sort! Just a LocalCluster on the same host
oh interesting; I’ve never personally experienced that warning, but I searched the dask repo and only saw it when there were networking issues 🧐
@Marvin archive “Dask add_client() got an unexpected keyword argument ‘versions’”