Kledia
11/25/2024, 6:42 PMMarvin
11/25/2024, 6:43 PMfrom_source
function when you are deploying with an S3Bucket storage in Prefect, you need to ensure the following steps are covered, depending on whether you're using Prefect 2.x or 3.x. Based on what I found, here's a summary of the relevant information:
### Prefect 2.x
For Prefect 2.x, the documentation indicates that you can use the prefect.yaml
file to set up deployments with cloud-provider storage like an AWS S3 bucket. You'll typically use steps like push_to_s3
and pull_from_s3
in prefect_aws.deployments.steps
to specify how the project code is uploaded and retrieved. You need to set the bucket name, folder, and any necessary credentials.
Check out the Prefect 2.x AWS S3 Integration Guide for complete instructions.
### Prefect 3.x
In Prefect 3.x, deployments also rely on a prefect.yaml
file. Here, you specify an S3Bucket
block with necessary authentication. When setting up your entrypoint:
- Ensure the S3 bucket storage block is configured with credentials (AWSCredentials
block) and available in your environment where the flow code is running.
- In your prefect.yaml
, specify the path and credentials necessary to access the S3 bucket.
Explore this further in the Prefect 3.x Deployment Documentation.
Who would have guessed that staying well-organized with YAML files would be the key to syncing code across the universe? Maybe even Marvin would crack a smile...