<@ULVA73B9P> what are possible use cases for fetch...
# ask-marvin
p
@Marvin what are possible use cases for fetching code from blob storage, eg s3 instead of git
m
thought for 51.7 seconds
Ah, let me share some key use cases and considerations for deploying flows from blob storage like S3 instead of Git: 1. Automated Code Distribution - When your CI/CD pipeline builds and packages your flows, it can upload them directly to blob storage - Useful for organizations that want to separate code artifacts from source control - Allows for versioned releases without Git tags 2. Air-Gapped or Private Environments - When your deployment environment can't access GitHub/GitLab - When you need to keep code entirely within your private infrastructure - Particularly relevant for highly regulated industries 3. Large Dependencies or Data Files - When your flows include large model files, data sets, or dependencies - Blob storage is often better suited for large binary files than Git - Keeps your Git repository lean while storing larger assets elsewhere 4. Dynamic Code Generation - When flows are generated programmatically or modified by automated processes - Easier to push to blob storage than managing Git commits - Useful for template-based flow generation Here's a basic example of how to deploy from S3 in Prefect 3.x:
Copy code
python
from prefect import flow
from prefect.filesystems import S3

# Create and register the S3 block
s3_block = S3(
    bucket_path="my-bucket/flows",
    credentials={
        "aws_access_key_id": "xxx",
        "aws_secret_access_key": "xxx"
    }
)
s3_block.save("my-s3-block")

# Deploy the flow from S3
@flow
def my_flow():
    print("Hello from S3!")

if __name__ == "__main__":
    my_flow.from_source(
        source="s3/my-s3-block/path/to/flow.py",
        name="s3-deployed-flow"
    ).deploy()
Trade-offs to Consider: Advantages: - Better handling of large files - More flexible access control - Can be cheaper for large-scale storage - Direct integration with cloud infrastructure Disadvantages: - Less version control functionality than Git - No built-in code review process - Additional infrastructure to manage - May require additional security considerations Remember that you'll need to set up appropriate authentication and permissions for your blob storage service, and ensure your deployment environment has access to it. For production environments, consider combining both approaches: use Git for version control and code review, then use automated processes to package and upload releases to blob storage for deployment.