Chris Leber
07/02/2021, 11:00 PMdb_kwargs
account, etc
dbt = DbtShellTask(
return_all=True,
profile_name="test",
environment="dev",
overwrite_profiles=True,
log_stdout=True,
helper_script="cd dbtProject",
log_stderr=True,
dbt_kwargs={
"type": "snowflake",
"account": "${ACCOUNT}",
...
I then retrieve the account name as a prefect secret, and set it as an env variable called ACCOUNT when I run the flow:
with Flow() as flow:
account = PrefectSecret("snowflake-account")
parse = dbt(
command=f"dbt run --models ./models/{database}",
env={"ACCOUNT": account}
)
I have used this approach with ShellTasks, and it works just fine. However, it is not working with DBTShellTasks. Instead of using the value of the Prefect Secret for the account name, it is trying to connect me to snowflake with "${ACCOUNT}" as the account name.
Any ideas?? Thanks in advance for the help 😁Kevin Kho
07/02/2021, 11:08 PMwith Flow() as flow:
account = PrefectSecret("snowflake-account")
parse = dbt(
command=f"dbt run --models ./models/{database}",
dbt_kwargs={
"type": "snowflake",
"account": account,
...
This will pass the value of the secret at runtimeChris Leber
07/02/2021, 11:51 PM