https://prefect.io logo
j

Joshua Hansen

09/05/2023, 5:43 PM
šŸ‘‹ I'm ramping up while assessing Prefect self-hosted for a project and have a question re: how to integrate deployments into CI/CD. Looking at this recipe i'm getting the sense that one would run this (lets say on merge) and any changes to code/deploy-config would overwrite the existing deployment by name? Trying understand how to structure the code and access. For example, in this blog post, the code is pushed to S3 and then operated on, which I don't want to do. I'd prefer the code stays in SCM and either is deployed directly to Prefect. Am I correct in my understandings here? What are other folks doing with their CI/CD setups?
n

Nate

09/05/2023, 8:42 PM
hi @Joshua Hansen - that recipe you linked is a bit old and leverages the
prefect deployment build
UX for block-based deployments I'd recommend using `prefect.yaml` to define your deployments and then
prefect deploy
something like this the example i just linked might be slightly overcomplicated (im checking for modified files and dynamically iterating over them) but in general you want to ā€¢ set your PREFECT_API_URL in the env so your CI machine can talk to the server ā€¢ run
prefect --no-prompt deploy myfile.py:my_flow_function
for each flow you want to deploy ( or
prefect --no-prompt deploy --all
to deploy them all) The
--no-prompt
part is important for CI because it means "turn off interactive mode"
run this (lets say on merge) and any changes to code/deploy-config would overwrite the existing deployment by name?
but yes this sounds right, your CI could generate a
version
(commonly a Git SHA) to attach to the deployment if you need to differentiate
šŸ‘ 1
j

Joshua Hansen

09/05/2023, 8:46 PM
I see. Thanks for the comments, I'll have a look through the links, but a quick glance it seems it'd be preferred to use the declarative yaml, be smart in the CI step about which things require deployment and go from there.
šŸ‘ 1
n

Nate

09/05/2023, 8:53 PM
a quick glance it seems it'd be preferred to use the declarative yaml, be smart in the CI step about which things require deployment and go from there.
I agree - this has been the easiest way in my experience, you can fully leverage the templating options in
prefect.yaml
and then handle your specific version control strategy / selective deployment within CI
2 Views