https://prefect.io logo
Title
f

Filip Panovski

03/23/2023, 3:25 PM
Hi everyone, I have a question with regard to organizing my deployments. I have two separate flow definitions, flow
A
and
B
. Each of these has potentially multiple deployments (for different parameters) and schedules. So for example:
A/process_with_param_set_1, daily @ 05:00
A/process_with_param_set_2, daily @ 05:00
B/process_daily, daily @ 07:00
B/process_monthly, monthly @ 7:15
I'm currently creating these deployments via the CLI. My problem is that this seems difficult to organize well. When I create the deployments
A/process_with_param_set_1
and
A/process_with_param_set_2
, it's already the case that I only have a single
A-deployment.yml
which is overwritten (I expected 2, one for each deployment of the flow). In addition, these always copy the entire directory structure, so it's difficult to have different code versions running in different deployments (for example:
A/process_with_param_set_1
based on commit sha X,
A/process_with_param_set_1
based on commit sha Y). Can anyone tell me what the recommended workflow for multiple deployments of a single flow is? Also, is there a way to not overwrite my deployment YML files when creating multiple deployment from the same flow? Should I have different entrypoints per deployment (since it seems that the deployment YML name is based on the entrypoint)?
1
s

Sahil Rangwala

03/23/2023, 6:21 PM
For more complex deployment interactions, you can programmatically create a deployment object so its easier to organize and automatically run. The cli is generally used for simpler one-off deployments where you can then configure with a .yml file
f

Filip Panovski

03/24/2023, 1:15 PM
Thanks for the feedback. That does seem to be the way to go for a maintainable deployment solution.