https://prefect.io logo
Title
t

Trip Humphrey

03/21/2022, 3:15 AM
Is there an example template of what a properly architected git repo looks like for a Prefect project? I am having trouble conceptualizing the development workflow. I am imagining a "production" branch that at any given moment contains a portfolio of all the relevant flows for a given project. Changes (i.e. merges) to the production branch would trigger incremental registrations of the modified flows driven by some sort of backend "build" job triggered by the successful merge. Other things I am considering would be a way to have proposed changes to production flows go through automated testing before being merged to production and ideally a way for developers to test their flows in development without sullying the production environment. Bonus points for being able to code the tasks and flows in such a way so that they do not require editing between environments. is there a recognized reference architecture for such a build strategy?
šŸ‘€ 1
t

Tomer Cagan

03/21/2022, 8:30 AM
Take a look here: https://github.com/PrefectHQ/prefect/discussions/4042 - I believe it has some relevant information
šŸ™ 1
:upvote: 2
a

Anna Geller

03/21/2022, 9:55 AM
By saying "a properly architected git repo" you are setting a high bar here and maybe no one will dare to share their repos šŸ˜„ But here are some I did in the past - I don't claim there are properly architected - just a structure that made sense for those projects: ā€¢ a general scaffold for code packaging https://github.com/anna-geller/packaging-prefect-flows/ ā€¢ https://github.com/anna-geller/flow-of-flows ā€¢ https://github.com/anna-geller/prefect-dbt-k8s-snowflake ā€¢ https://github.com/anna-geller/prefect-monte-carlo/tree/workshop To be clear, those are all for Prefect 1
if you are asking about CI/CD with Prefect 1, check out those resources https://discourse.prefect.io/tag/ci-cd
t

Trip Humphrey

03/21/2022, 12:28 PM
@Anna Geller @Tomer Cagan Both much appreciated. This is precisely the sort of info I am looking for.
@Anna Geller Re: the "proper" comment I definitely didn't mean to imply that..lol. I just meant better than my currently floundering attempts. šŸ˜…
šŸ‘ 1
k

Kevin Kho

03/21/2022, 2:46 PM
I was looking through Gtihub search before and found this. Seems like they use this package for all of the task logic and then build Flows with this as a dependency