Hi,
I am trying to write a flow that uses boto3 for interacting with AWS resources, i am installing prefect with aws extra in the docker image,
but the flow fails in tasks that try to create aws clients with:
“`botocore.exceptions.DataNotFoundError: Unable to load data for: endpoints`”
any suggestions ?
k
Kevin Kho
08/11/2021, 10:28 PM
Hey @Nadav, how do you install it with the aws extra?
pip install
or
conda
? And then are you using a DaskExecutor?
n
Nadav
08/11/2021, 10:55 PM
@Kevin Kho i install with pip and i do not use dask i use the default, i guess that is local right?
k
Kevin Kho
08/11/2021, 10:58 PM
Yeah that would be local. I haven’t seen this before. What boto3 version do you get?
Kevin Kho
08/11/2021, 11:01 PM
Maybe you can also try explicitly installing boto3 (I don’t think it’s the issue. I am more assuming that an upgrade is needed)
n
Nadav
08/11/2021, 11:10 PM
@Kevin Kho the boto3 version is .18.19,
i tried installing boto3 using the requirements file but i get the same error
Nadav
08/11/2021, 11:18 PM
when i run the docker container with -it and i run python importing boto3 and creating a client is succeeded
k
Kevin Kho
08/12/2021, 12:01 AM
Are you hosting that container on AWS then using it in your flow?
n
Nadav
08/12/2021, 5:31 AM
Yes the flow storage is docker hosted on ecr
k
Kevin Kho
08/13/2021, 1:02 AM
Oh sorry I didn’t get back to you. But I would just make sure you pull the latest image from ECR? Like maybe tag it? Because if it worked in the container locally, it should work on ECR as well.