Josh Greenhalgh

    Josh Greenhalgh

    1 year ago
    Has anyone got any thoughts on testing deployment process? I will be running my flows on k8s cluster but would like devs to be able to test out their flows locally before submitting a PR which will deploy the changed flow to the server - it's not completely clear to me what the best process is here so would be good to hear of other peoples approaches?
    b

    Brian Mesick

    1 year ago
    We are getting ready to start a CI/CD process for our Prefect Flows and common utilities, but are in the same boat of it not being very clear how / if things should be tested.
    Michael Adkins

    Michael Adkins

    1 year ago
    I’m slowly putting together some best-practices around this but there isn’t a clear consensus yet! Reach out if you have an elegant solution or general questions 🙂
    Josh Greenhalgh

    Josh Greenhalgh

    1 year ago
    Any chance of sneek peek at what you have so far?
    I really just want a sense of the landscape of options...
    Michael Adkins

    Michael Adkins

    1 year ago
    Unfortunately not, in my free time I’m overhauling an internal repo of flows that we use at Prefect to run our org and I don’t have very much free time so it’s going slow 😄
    Let me look into starting a collaborative document so we can list options
    Josh Greenhalgh

    Josh Greenhalgh

    1 year ago
    That would be great!
    b

    Brian Mesick

    1 year ago
    I’d love to get in on that too!
    Billy McMonagle

    Billy McMonagle

    1 year ago
    very interested in this as well
    Michael Adkins

    Michael Adkins

    1 year ago
    I’m working on a template right now 🙂
    What are your thoughts on these headers?
    ## Overview
    *A bullet point list describing the process of flow development -> deployment -> run*
    
    ## Project structure
    *An outline of your project file structure and brief discussion*
    
    ## Developing flows
    
    ### Writing flows
    *Where are flows written? Do they share common patterns?*
    
    ### Testing flows
    *How flows are tested*
    
    ## Deploying flows 
    
    ### Registering flows
    *How flows are registered. How are the flows discovered? How are options determined?*
    
    ### Storing flows
    *What flow storage do you use? Why?*
    
    ## Running flows
    
    ### Run config
    *What run config do you use for your flows? How do you set it?*
    
    ## Limitations
    *Downsides to your approach*
    
    ## Benefits
    *Benefits to your approach*
    Billy McMonagle

    Billy McMonagle

    1 year ago
    I think it might be broadly helpful to have a breakdown of the difference between Build, Deploy, and Register. I don't see the term "Build" in this list so I'm guessing "Deploy" covers that stage.
    If folks are interested in comparing their experiences (with code samples) for each of these subject areas, I'd be glad to jump in on that. Maybe even good to have a channel focused on this discussion topic?
    Michael Adkins

    Michael Adkins

    1 year ago
    Billy McMonagle

    Billy McMonagle

    1 year ago
    Thanks @Michael Adkins. I am going to take some time and write up my approach... I'm at the early stages so this is a helpful framework to think through some still fuzzy details.