Trip Humphrey

    Trip Humphrey

    6 months ago
    Is there an example template of what a properly architected git repo looks like for a Prefect project? I am having trouble conceptualizing the development workflow. I am imagining a "production" branch that at any given moment contains a portfolio of all the relevant flows for a given project. Changes (i.e. merges) to the production branch would trigger incremental registrations of the modified flows driven by some sort of backend "build" job triggered by the successful merge. Other things I am considering would be a way to have proposed changes to production flows go through automated testing before being merged to production and ideally a way for developers to test their flows in development without sullying the production environment. Bonus points for being able to code the tasks and flows in such a way so that they do not require editing between environments. is there a recognized reference architecture for such a build strategy?
    t

    Tomer Cagan

    6 months ago
    Take a look here: https://github.com/PrefectHQ/prefect/discussions/4042 - I believe it has some relevant information
    Anna Geller

    Anna Geller

    6 months ago
    By saying "a properly architected git repo" you are setting a high bar here and maybe no one will dare to share their repos šŸ˜„ But here are some I did in the past - I don't claim there are properly architected - just a structure that made sense for those projects: ā€¢ a general scaffold for code packaging https://github.com/anna-geller/packaging-prefect-flows/ ā€¢ https://github.com/anna-geller/flow-of-flows ā€¢ https://github.com/anna-geller/prefect-dbt-k8s-snowflake ā€¢ https://github.com/anna-geller/prefect-monte-carlo/tree/workshop To be clear, those are all for Prefect 1
    if you are asking about CI/CD with Prefect 1, check out those resources https://discourse.prefect.io/tag/ci-cd
    Trip Humphrey

    Trip Humphrey

    6 months ago
    @Anna Geller @Tomer Cagan Both much appreciated. This is precisely the sort of info I am looking for.
    @Anna Geller Re: the "proper" comment I definitely didn't mean to imply that..lol. I just meant better than my currently floundering attempts. šŸ˜…
    Kevin Kho

    Kevin Kho

    6 months ago
    I was looking through Gtihub search before and found this. Seems like they use this package for all of the task logic and then build Flows with this as a dependency