Santiago Toso
11/11/2022, 4:54 PMbotocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
. It is uploading the file correctly, and even the image is building correctly but it is failing when I run the flow from the UI. From what I understand in the error message, when running the flow from the docker image, it can't access S3 to read the code. Here are a few more details:
• Storage block --> S3. It uploads correctly
• Docker image --> Custom image retrieved from ECR
• Code --> It is the same code I use in other non-dockerized deployment.
• The EC2 instances for the Prefect Agent, Prefect Server and S3 Bucket are all in the same AWS account
• The EC2 instance for the PrefectAgent has S3 full access (should the EC2 instance for the Prefect Server also have full S3 access?)
I leave in the image below with the full error I get in the Prefect UI and in the Agent's terminal.
I saw a similar AccessDenied
error in this channel when trying to upload the code to S3 but this seems different. The problem is when the flow is running in docker and can't retrieve the code.
Any ideas why this could be? Thanks a lot for your help!Christopher Boyd
11/11/2022, 4:56 PMSantiago Toso
11/11/2022, 4:58 PMsts.getidentity()
to my code? Could you point me to the API reference of this so I can take a deeper look?Christopher Boyd
11/11/2022, 5:01 PMSantiago Toso
11/11/2022, 5:02 PMChristopher Boyd
11/11/2022, 5:03 PMSantiago Toso
11/11/2022, 5:03 PMChristopher Boyd
11/11/2022, 5:04 PMimport boto3
s3 = boto3.resource('s3')
s3.Bucket('mybucket').download_file('hello.txt', '/tmp/hello.txt')
Santiago Toso
11/11/2022, 5:05 PMChristopher Boyd
11/11/2022, 5:06 PMin order to have access to objects within a bucket you need a /*
Tobias
11/14/2022, 7:43 AM--upload
there can be issues
https://prefect-community.slack.com/archives/C048K0MGHNK/p1667990890478769Santiago Toso
11/14/2022, 8:14 AMAccessDenied
error when running it as a dockerized deployment.
I'll try deploying my image as a Lambda function to see if my Docker file has a problem. If it runs correctly I could only think the problem has to do with Prefect. Does this sound like a right conclusion to you?Christopher Boyd
11/14/2022, 1:08 PMPekka
11/15/2022, 6:58 AM403 Forbidden
when the Prefect Orion server (docker image) was not the correct newest version w.r.t. to the clientSantiago Toso
11/15/2022, 3:50 PMChristopher Boyd
11/15/2022, 4:11 PMSantiago Toso
11/15/2022, 5:25 PMPekka
11/16/2022, 6:11 AMChristopher Boyd
11/16/2022, 2:58 PM