Resurrecting https://prefect-community.slack.com/archives/CL09KU1K7/p1616759310466600 When I have the following in my requirements.in file:

boto3
dask-cloudprovider[aws] == 2021.3.1
prefect[aws] == 0.14.17

Trying to run pip-compile on it fails with a complaint about botocore versioning:

botocore<1.20.50,>=1.20.49 (from aiobotocore==1.3.0->dask-cloudprovider[aws]==2021.3.1->-r requirements/production.in (line 14))
  botocore<1.21.0,>=1.20.67 (from boto3==1.17.67->-r requirements/production.in (line 9))
  botocore<2.0a.0,>=1.12.36 (from s3transfer==0.4.2->boto3==1.17.67->-r requirements/production.in (line 9))

This boils down to: botocore ==1.20.49,>=1.20.67 which cannot be satisfied. What is a safe version of boto3 that works with prefect and dask cloud provider? Removing the version numbers above from the requirements.in does not help, same complaints. Which is to be expected since pip is going to try the newest it can without guidance.

I think we ended up with:

botocore = "==1.19.52"
dask-cloudprovider = {version = "==2021.3.1", extras = ["aws"]}
boto3 = "==1.16.52"

That is a helpful start. The previous thread did not have a satisfying conclusion.

https://github.com/pangeo-forge/pangeo-forge-aws-bakery/blob/main/Pipfile.lock#L500 we have 1.2.2 of

aiobotocore

FUN

🎉

I was look through that thread as well when I wanted to try spinning up a dask cluster in a flow. I use Pipenv and managed to get it working with these packages and pins:

[packages]
aiobotocore = {extras = ["boto3"], version = "*"}
dask-cloudprovider = {extras = ["aws"], version = "*"}
prefect = {extras = ["aws"], version = "*"}

It installs/locks successfully and ran well a couple of weeks ago with Prefect 0.14.16