https://prefect.io logo
Title
l

Leon Kozlowski

02/22/2022, 9:28 PM
Does the DbtShellTask work with the open-lineage dbt? (
dbt-ol run
)
🙌 1
a

Anna Geller

02/22/2022, 9:31 PM
I'd say no but to be sure you would need to give it a try cc @alex
a

alex

02/22/2022, 9:34 PM
It probably wouldn’t work out of the box since it looks like there are some env variables that
dbt-ol
expects that the
DbtShellTask
does not set. You could subclass
ShellTask
and set the necessary env variables to run
dbt-ol
:upvote: 1
l

Leon Kozlowski

02/22/2022, 9:46 PM
Cool - I’ll try that
Update, it did work with
DbtShellTask
Heres the code sample
with Flow("dbt-transformations") as flow:
    dbt = DbtShellTask(
        return_all=True,
        log_stdout=True,
        log_stderr=True,
        profile_name="transformations",
        environment="dev",
        helper_script="cd transformations",
        profiles_dir="",
    )

    dbt_deps = dbt(task_args={"name": "DbtDeps"}, command="dbt deps")

    dbt_doc_generate = dbt(
        task_args={"name": "DbtDocGenerate"},
        command="dbt docs generate",
        upstream_tasks=[dbt_deps],
    )

    dbt(
        task_args={"name": "DbtOpenLineageRun"},
        command="dbt-ol run",
        upstream_tasks=[dbt_doc_generate],
    )