Quick question on the Dask Cluster tutorial: Getting the following error when the flow tries to run

distributed.core - INFO - Starting established connection
distributed.core - ERROR - add_client() got an unexpected keyword argument 'versions'
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/distributed/core.py", line 412, in handle_comm
    result = handler(comm, **msg)
TypeError: add_client() got an unexpected keyword argument 'versions'

With

dask-scheduler, version 2.2.0

and

Prefect, version 0.9.0

. Makes me think a version issue?

Hey @Chris O'Brien - this is caused by a version mismatch between the dask-distributed version that your Flow is using and the version of dask that the cluster is using — where are you spinning this cluster up?

Ah I had observed this earlier. Looks to be a version issue w/ distributed after this recent change https://github.com/dask/distributed/commit/32cb96effe9287fdc42dd73be36989be13abe99d#diff-6a3108098afcddf88b38652ae1e383b5 The

distributed

version in the prefecthq/prefect docker image is

2.9.1

but the version of distributed that your flow is using is probably higher. Are you building your flow with a Docker storage?

Ahhhhh, so for my Prefect project it has dask=2.9.2

"dask": {
           ...
            "version": "==2.9.2"
        },

Thinking I should downgrade this?

yea you could downgrade that to

2.9.0

and it should begin working; you could also build your own custom image to use within the cluster if you use any 2.9.2-specific features

Ooooooof, sometimes Python is a hell of a time. So got the communication working and just working through some exceptions now!

Hmmm that suggests that your dask cluster is having a networking problem of some kind

Heh, nothing of the sort! Just a LocalCluster on the same host

oh interesting; I’ve never personally experienced that warning, but I searched the dask repo and only saw it when there were networking issues 🧐

@Marvin archive “Dask add_client() got an unexpected keyword argument ‘versions’”