https://prefect.io logo
#prefect-community
Title
# prefect-community
j

Justin Burchard

04/21/2022, 7:25 PM
Hello everyone. I'm a recent adopter of Prefect 1.0. I have a use case in which I have a flow that I would like to reuse for multiple projects, with different credentials and database schemas. I'd like to use a config file to publish the flows. It seems like Parameters would be great for this, but the documentation seems to suggest can only assign them for specific flow runs. Does anyone have any suggestions as to how I could structure this?
k

Kevin Kho

04/21/2022, 7:35 PM
You can Parameterize the Secret name to pull to create the Database connection. If you want to do it as a Flow, you can do
Copy code
with Flow('parent') as flow:
   create_flow_run(flow_name=.., project_name=..., parameters={'env': "DEV"})
and then in the child flow:
Copy code
@task
def get_secret_name(env):
    return env+"_DATABASE"
with Flow('child') as flow:
    env = Parameter('env')
    secret_name = get_secret_name(env)
    connection = PrefectSecret()(secret_name)
    ....
If you want to reuse a task across flows:
Copy code
@task
def get_db(env):
    connection = Secret(env+"DATABASE").get()
    do_something(connection)

with Flow("...") as flow:
    env = Parameter('env')
    get_db(env)
so here the Secret is pulled inside the task and the Parameter can be changed on the Flow level. Does that give ideas?
j

Justin Burchard

04/21/2022, 7:38 PM
Yes, that is great. Thank you!
5 Views