Leon Kozlowski
02/22/2022, 9:28 PMdbt-ol run
)Anna Geller
02/22/2022, 9:31 PMalex
02/22/2022, 9:34 PMdbt-ol
expects that the DbtShellTask
does not set. You could subclass ShellTask
and set the necessary env variables to run dbt-ol
Leon Kozlowski
02/22/2022, 9:46 PMDbtShellTask
with Flow("dbt-transformations") as flow:
dbt = DbtShellTask(
return_all=True,
log_stdout=True,
log_stderr=True,
profile_name="transformations",
environment="dev",
helper_script="cd transformations",
profiles_dir="",
)
dbt_deps = dbt(task_args={"name": "DbtDeps"}, command="dbt deps")
dbt_doc_generate = dbt(
task_args={"name": "DbtDocGenerate"},
command="dbt docs generate",
upstream_tasks=[dbt_deps],
)
dbt(
task_args={"name": "DbtOpenLineageRun"},
command="dbt-ol run",
upstream_tasks=[dbt_doc_generate],
)