https://prefect.io logo
Title
n

Nataliia Sherstneva

03/16/2023, 3:24 PM
Hi prefect community, I have a model that was trained with Prefect workflow and saved in AWS S3 Bucket.
def upload_dir_to_s3(local_model_dir, logger, target_model_dir=None):
    s3_bucket = S3Bucket.load(BLOCK_NAME_IRS_TRAIN_DATA_S3)

    if not target_model_dir:
        target_model_dir = local_model_dir
    try:
        s3_bucket.upload_from_folder(from_folder=local_model_dir, to_folder=f"{target_model_dir}")
        <http://logger.info|logger.info>(f"Success! Directory '{local_model_dir}' was uploaded to '{s3_bucket.bucket_name}' Bucket.")
    except Exception:
        raise IOError(f"Saving to '{s3_bucket.bucket_name}' Bucket Failed!")
Then I download this model from S3 Bucket, however not within Prefect Workflow, but separately with
boto3.client
. Locally it works, but in K8s Cluster I get an Forbidden Error.
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
Could you please tell me, what the problem might be? Should I save or read the model in another way?