Hi. My question is related to prefect-dask collection. Depends on the image I use for my DaskCluster Prefect submits tasks to DaskTaskRunner with different behavior. My task does manipilations with dask.dataframe and in one case dask transformation is distributed on the same cluster, but in other case task is running on one worker and other just waiting. Does anyone has an idea what should be different to see such different behaviour? Thanks.

@Andrew Huang Do you have any insights into this?

Maybe https://github.com/PrefectHQ/prefect-dask#distributing-dask-collections-across-workers

Thanks, didn't know about it. I also found what is difference between my 2 versions. It distribute the work in the case where I register PipInstall plugin. Sounds weird, but that is how it works.