Hey team - whats the best practice for building a ...
# prefect-community
Hey team - whats the best practice for building a deployment for a package with multiple flows and pushing to a storage ie s3?
currently I have a python package with a few flows, so I build a deployment for each and apply, but I end up uploading the package data to my storage block multiple times
and is it possibly to push to a subpath of the storage block? Say I have a storage block in s3:
and I want to push
here - is there a way I can push the build artifacts to ``s3://brad/flows/flow1/ and s3://brad/flows/flow2/`
At the moment it seems like I would need to create a storage block per flow (otherwise they will clobber each other)? Is that correct
I'm also noticing this weird behaviour. Building a deployment uploads all files under its current directory, so unrelated flows also get overwritten. One workaround is to create a block for each developer and environment, and let them manage their storage on their own to avoid overwriting others' files.
yeah - but then its one block per flow right @Pipat (Benz) Methavanitpong ? TRhats not going to scale well (I store all my flows in a single repo)
I meant a block per developer, and a developer uploads a whole repo into his block. Because we can't avoid uploading unrelated flows in a deployment. This should make accidentally overwriting other flows more manageable.
@Brad I think it makes more sense to have one infra block (e.g. KubernetesJob block) per Dockerized environment and one storage block (e.g. S3 block) per branch (e.g. main / staging) per repo
Then multiple flows with the same Docker image in the same Python subpackage can be associated with 1 infra block and 1 storage block accordingly.
I agree @Chris L. - but if I want to generate a deployment yaml for each flow, then I end up re-uploading the build artifact every time right?
Yeah…but hopefully the new local storage for Docker resolves this issue. Or maybe something similar to git storage in 1.0
yeah git storage would for sure solve my issue