Hello. I'm using `subprocess` to run my dbt projec...
# prefect-community
m
Hello. I'm using
subprocess
to run my dbt project locally (Prefect 1.2.2, Windows). I create my command (
dbt run --vars '{data_processing_start_date: 2022-07-20, data_processing_end_date: 2022-07-20}' --profiles-dir ./
) like this:
Copy code
command = (
        f"dbt run --vars '{{"
        f"data_processing_start_date: {data_processing_start_date}, "
        f"data_processing_end_date: {data_processing_end_date}}}' --profiles-dir ./ "
    )
The command is created correctly, but dbt gives me this error
dbt: error: unrecognized arguments: 2022-07-20, data_processing_end_date: 2022-07-20}'
Seems like dbt interprets
2022-07-20
as an argument instead of the value for
data_processing_start_date
variable. Keep in mind that when I run the same command in my CLI, dbt works fine. But when it's provided to dbt through
subprocess
this occurs. This is my subprocess:
Copy code
subprocess.run(
        command,
        check=True,
        stderr=True,
        stdout=True,
        shell=True,
        cwd="dbt",
    )
Any idea what might cause this, and how to solve this? Thank you
1
a
Can you try using dbt task instead? in 1.0, we have https://docs.prefect.io/api/latest/tasks/dbt.html and in 2.0 https://github.com/PrefectHQ/prefect-dbt
using this task, you don't even have to specify the profiles directory manually if you need more examples, we have a blog post series about it https://www.prefect.io/guide/blog/flow-of-flows-orchestrating-elt-with-prefect-and-dbt/
m
The reason I'm using subprocess is that Prefect Shell Task is not compatible with windows
a
oh I'm sorry, I missed that part, you're right - it's more complicated then...
👍 1
124 Views