Piotr
07/01/2025, 5:19 PMMarvin
07/01/2025, 5:19 PMMarvin
07/01/2025, 5:20 PMpython
from prefect import flow
from prefect.filesystems import S3
# Create and register the S3 block
s3_block = S3(
bucket_path="my-bucket/flows",
credentials={
"aws_access_key_id": "xxx",
"aws_secret_access_key": "xxx"
}
)
s3_block.save("my-s3-block")
# Deploy the flow from S3
@flow
def my_flow():
print("Hello from S3!")
if __name__ == "__main__":
my_flow.from_source(
source="s3/my-s3-block/path/to/flow.py",
name="s3-deployed-flow"
).deploy()
Trade-offs to Consider:
Advantages:
- Better handling of large files
- More flexible access control
- Can be cheaper for large-scale storage
- Direct integration with cloud infrastructure
Disadvantages:
- Less version control functionality than Git
- No built-in code review process
- Additional infrastructure to manage
- May require additional security considerations
Remember that you'll need to set up appropriate authentication and permissions for your blob storage service, and ensure your deployment environment has access to it.
For production environments, consider combining both approaches: use Git for version control and code review, then use automated processes to package and upload releases to blob storage for deployment.