Justin Burchard

    Justin Burchard

    5 months ago
    Hello everyone. I'm a recent adopter of Prefect 1.0. I have a use case in which I have a flow that I would like to reuse for multiple projects, with different credentials and database schemas. I'd like to use a config file to publish the flows. It seems like Parameters would be great for this, but the documentation seems to suggest can only assign them for specific flow runs. Does anyone have any suggestions as to how I could structure this?
    Kevin Kho

    Kevin Kho

    5 months ago
    You can Parameterize the Secret name to pull to create the Database connection. If you want to do it as a Flow, you can do
    with Flow('parent') as flow:
       create_flow_run(flow_name=.., project_name=..., parameters={'env': "DEV"})
    and then in the child flow:
    @task
    def get_secret_name(env):
        return env+"_DATABASE"
    with Flow('child') as flow:
        env = Parameter('env')
        secret_name = get_secret_name(env)
        connection = PrefectSecret()(secret_name)
        ....
    If you want to reuse a task across flows:
    @task
    def get_db(env):
        connection = Secret(env+"DATABASE").get()
        do_something(connection)
    
    with Flow("...") as flow:
        env = Parameter('env')
        get_db(env)
    so here the Secret is pulled inside the task and the Parameter can be changed on the Flow level. Does that give ideas?
    Justin Burchard

    Justin Burchard

    5 months ago
    Yes, that is great. Thank you!