hi everyone! i'm attempting to get DBT shell tasks...
# prefect-community
r
hi everyone! i'm attempting to get DBT shell tasks running via prefect. attaching my code below. anyone have any ideas on what i'm getting wrong? i can confirm that the
profiles.yml
was created correctly. im able to run dbt commands with the newly created
profiles.yml
outside of the prefect package
conversely, this does run. does anyone know how to check the dbt logs for this?
h
Hi Raghav, In your first example, you define the dbt task by instantiate the task class. In that way, you need to use task init(). then you do task run() in the flow.
so try this in your first example,
Copy code
marketing_task = dbtTask(
… # except command param
)
With Flow() as flow:
   marketing_task( command= "your dbt command")
r
hi Hui, thanks for reaching out! still no luck. i attempted
marketing_task.run('command')
within the flow as well
j
I had a similar issue, I don't recall the error message so it may be unrelated, but I ended up using the full directory path.
r
i don't think the issue is with the directory. i attempted to run a simple shell task (
cd ~
). i have a feeling prefect is unable to access my shell. i am working on windows so wondering if that may be a problem
j
This works for on a Mac running locally: profile_directory='/Users/****/development/dbt_projects/sf_sandbox/.dbt' with Flow(name="dbt_flow") as f: task = DbtShellTask( profile_name='sf_sandbox', environment='dev', overwrite_profiles=True, profiles_dir=profile_directory, set_profiles_envar=True, log_stderr=True, return_all=True, dbt_kwargs={ 'type': 'snowflake', 'account': '********.east-us-2.azure', # User/password auth 'user': '***********', 'password': '********', 'role': '******', 'database': '******', 'warehouse': 'COMPUTE_WH', 'schema': 'DBT' } )(command='dbt run --project-dir=/Users/********/development/dbt_projects/sf_sandbox')
r
@James Parham tried with the full directory. still no luck 😞
h
that’s interesting. try to print the error log from DBTshellTask to see why it fails. they recently enabled that . see this issue and the its resolution.
Copy code
<https://github.com/PrefectHQ/prefect/issues/3584>