Hello all, Despite reading the various posts about the "Access Denied" S3 error ; I can't get my deployment to upload to S3 from my local laptop 😞.

botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

+

PermissionError: Access Denied

when doing

prefect deployment build flow-example.py:hi -q default -n flow-example-aws-d --apply -sb s3/aws-personal-s3

I am using a IAM user key/secret, it works well for uploading files via AWS CLI, and Prefect Cloud Blocks (named . As I'm discovering both Prefect Blocks and AWS S3/IAM management ; I have a few questions: • are there constraints regarding the S3 buckets ? (regions, public access ?) • do I need both AWS Credential and S3 blocks created in ? • do I need to use manually-created nested blocks ? (but I don't see the point) • do I need to use IAM roles (which I haven't studied yet 😇 ) boto* libs

- boto3=1.26.50
- botocore=1.29.50
- aiobotocore==2.4.2
- mypy-boto3==1.26.50
- mypy-boto3-s3==1.26.0.post1

prefect* version

prefect==2.7.8+4.ge291d4c18
prefect-aws==0.2.3

Hi @Clément Peyrard, has the IAM user the AmazonS3FullAccess permission?

Yes, it belongs to a User Group that has it, and it looks like they flow correctly to the user (PrefectS3BucketPolicy is an attempt to add all kms rights)

Did you set up the S3 block from the UI or with a script?

From the UI ; tried from a script as well (from AWSCredentials as recommended by @Bianca Hoch in https://prefect-community.slack.com/archives/CM28LL405/p1669831027116889 ) but no success - and I don't understand this recommendation.

You have created the bucket, don't you?

Yes, I manually created it beforehand - should I let prefect create it ?

No, I think is ok

Hi @Clément Peyrard - sorry you've been running into trouble does your s3 bucket block (

s3/aws-personal-s3

) have an

AWSCredentials

block attached to it? I wonder if your machine is using a separate set of aws credentials when trying to upload the deployment files

should I let prefect create it ?

sounds like you already have a bucket, but just a heads up - currently prefect will not create infrastructure like s3 buckets on your behalf, blocks like

S3Bucket

are just meant to represent existing resources out in the world (like AWS), and allow you to connect and interact with them

Hi @Nate, thanks for the reply. At first I did not "attached" an AWSCredentials (I thought a S3bucket block could exist on its own but I'm a bit unclear on this given past responses and doc). But I also did a test following the link I shared and I was not successful by creating a manual S3 block. • Is it mandatory to have a AWSCredentials block ? (and if so, is it mandatory to provide a

Aws Session Token

at block creation ?) • Is there a nice way to check which authentication is used at upload ?

I didn't implement this, but from looking here (which is ultimately how we connect you to s3 when you build a deployment that will be stored in s3) it seems that we'll first attempt to use

AWSCredentials

stored on the

S3Bucket

block to create a boto client, and if there aren't any credentials on the block, we'll defer to boto (which I believe will try to pull from your default credentials in

~/.aws/credentials

) so while I don't think its strictly necessary to set

AWSCredentials

on your

S3Bucket

, I think it may be useful to try, if only to eliminate some potential causes of this error