Hi, Is it possible to override db_name, user and host values in the runtime for postgres task? ```po...

Maria

12/10/2021, 6:32 AM

Hi, Is it possible to override db_name, user and host values in the runtime for postgres task?

Copy code

postgres_execute = PostgresExecute(    db_name="abc", user="peter", host="my_host", port=5432,)

I have multiple databases (dev, test, demo, +prod db per client), and I need to query/load data into the relevant one. I have a config file and I was hoping I could use Parameter to tell what config line to choose - but it seems like I cannot override those params... What is the recommended approach for such use case?

Anna Geller

12/10/2021, 10:11 AM

@Maria are you on Prefect Cloud? if so, you could leverage the KV Store for this purpose.

Anna Geller

12/10/2021, 10:18 AM

but the easiest would be if you create your own version of Postgres task if you need those attributes to be modified at runtime - here is an example of how this could look like (I haven’t tested this yet) https://gist.github.com/c272f7b6661895f06c5681cacc23a1c0

Maria

12/10/2021, 10:18 AM

I am. Looking into it now, how can I leverage this?

Anna Geller

12/10/2021, 10:19 AM

probably the second option with the Gist will be easier and cleaner than KV store

Maria

12/10/2021, 10:20 AM

Right I see. Yes I think I understand how to use gist option. Thanks a lot!!

🙌 1

Kevin Kho

12/10/2021, 2:53 PM

Anna is right. It would be easier to use a modified version of the postgres task

3 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.