https://prefect.io logo
Title
d

David Cupp

09/28/2022, 4:20 PM
Hello, I have high level design question: It is advisable to share the flow code between two deployments of the same flow? details in thread
1
It looks like I could do this with the Python API:
storage_block = S3.load("myfirstblock")

deploymentA = Deployment(
    flow_name="FooFlow",
    name="FooA",
    parameters={"arg", "A"}
    entrypoint="foo.py:foo"
    storage=storage_block,
    path="/foo_code"
)
deploymentA.apply()
deploymentA.upload_to_storage()

deployment = Deployment(
    flow_name="FooFlow",
    name="FooB",
    parameters={"arg", "B"}
    entrypoint="foo.py:foo"
    storage=storage_block,
    path="/foo_code"
)
deploymentB.apply()
k

Khuyen Tran

09/28/2022, 4:22 PM
Do you mean to ask if it is desirable to have two deployments for the same flow?
d

David Cupp

09/28/2022, 4:22 PM
I am storing code in s3 for now. I am going to have, for example, 100-200 deployments per flow (each with different arguments) so I would like to avoid making a separate upload for each deployment. Is re-using code like this one of the intended ways Deployments are used, or am I doing off the rails?
it is desirable to have two deployments for the same flow?
I'm asking specifically about whether its acceptable to have two deployments share the exact same chunk of flow code stored in S3.
From the docs it seems like that having multiple deployments per Flow registration is by design.
k

Khuyen Tran

09/28/2022, 4:24 PM
Yes, deployment allows you to run a single flow with different parameters as explained here
d

David Cupp

09/28/2022, 4:25 PM
Thanks, I read that. That is not what I'm asking.
k

Khuyen Tran

09/28/2022, 4:25 PM
However, you can create one deployment then override the default parameter with a custom parameter when running that deployment
d

David Cupp

09/28/2022, 4:26 PM
I would like to have two deployments (for the same flow), each with separate sets of default args (that may be overridden), but only upload the code to s3 once.
k

Khuyen Tran

09/28/2022, 4:27 PM
You can skip uploading when building a deployment using the command
--skip-upload
d

David Cupp

09/28/2022, 4:29 PM
Great. Just wanted to make sure I wasn't violating an implicit design rule. Thanks.