s

    Sean Perry

    1 year ago
    Resurrecting https://prefect-community.slack.com/archives/CL09KU1K7/p1616759310466600 When I have the following in my requirements.in file:
    boto3
    dask-cloudprovider[aws] == 2021.3.1
    prefect[aws] == 0.14.17
    Trying to run pip-compile on it fails with a complaint about botocore versioning:
    botocore<1.20.50,>=1.20.49 (from aiobotocore==1.3.0->dask-cloudprovider[aws]==2021.3.1->-r requirements/production.in (line 14))
      botocore<1.21.0,>=1.20.67 (from boto3==1.17.67->-r requirements/production.in (line 9))
      botocore<2.0a.0,>=1.12.36 (from s3transfer==0.4.2->boto3==1.17.67->-r requirements/production.in (line 9))
    This boils down to: botocore ==1.20.49,>=1.20.67 which cannot be satisfied. What is a safe version of boto3 that works with prefect and dask cloud provider? Removing the version numbers above from the requirements.in does not help, same complaints. Which is to be expected since pip is going to try the newest it can without guidance.
    ciaran

    ciaran

    1 year ago
    I think we ended up with:
    botocore = "==1.19.52"
    dask-cloudprovider = {version = "==2021.3.1", extras = ["aws"]}
    boto3 = "==1.16.52"
    Thought we haven't updated in a few weeks so that may have changed!
    s

    Sean Perry

    1 year ago
    That is a helpful start. The previous thread did not have a satisfying conclusion.
    aibotocore appears to be the sticking point. it wants 1.20.49 if botocore.
    ciaran

    ciaran

    1 year ago
    We got to a point where the order which you installed the packages made a difference.
    s

    Sean Perry

    1 year ago
    FUN
    ciaran

    ciaran

    1 year ago
    🎉
    Noah Holm

    Noah Holm

    1 year ago
    I was look through that thread as well when I wanted to try spinning up a dask cluster in a flow. I use Pipenv and managed to get it working with these packages and pins:
    [packages]
    aiobotocore = {extras = ["boto3"], version = "*"}
    dask-cloudprovider = {extras = ["aws"], version = "*"}
    prefect = {extras = ["aws"], version = "*"}
    It installs/locks successfully and ran well a couple of weeks ago with Prefect 0.14.16