Channels
#prefect-community
Title
r
Raghav Thirumulu
12/15/2020, 6:00 PMhi everyone! i'm attempting to get DBT shell tasks running via prefect. attaching my code below. anyone have any ideas on what i'm getting wrong? i can confirm that the
profiles.ymlprofiles.ymlconversely, this does run. does anyone know how to check the dbt logs for this?
h
Hui Zheng
12/15/2020, 6:16 PMHi Raghav, In your first example, you define the dbt task by instantiate the task class. In that way, you need to use task init().
then you do task run() in the flow.
so try this in your first example,
Copy code
marketing_task = dbtTask(
… # except command param
)
With Flow() as flow:
marketing_task( command= "your dbt command")r
Raghav Thirumulu
12/15/2020, 6:23 PMhi Hui, thanks for reaching out! still no luck. i attempted
marketing_task.run('command')j
James Parham
12/15/2020, 7:30 PMI had a similar issue, I don't recall the error message so it may be unrelated, but I ended up using the full directory path.
r
Raghav Thirumulu
12/15/2020, 7:34 PMi don't think the issue is with the directory. i attempted to run a simple shell task (
cd ~j
James Parham
12/15/2020, 7:41 PMThis works for on a Mac running locally:
profile_directory='/Users/****/development/dbt_projects/sf_sandbox/.dbt'
with Flow(name="dbt_flow") as f:
task = DbtShellTask(
profile_name='sf_sandbox',
environment='dev',
overwrite_profiles=True,
profiles_dir=profile_directory,
set_profiles_envar=True,
log_stderr=True,
return_all=True,
dbt_kwargs={
'type': 'snowflake',
'account': '********.east-us-2.azure',
# User/password auth
'user': '***********',
'password': '********',
'role': '******',
'database': '******',
'warehouse': 'COMPUTE_WH',
'schema': 'DBT'
}
)(command='dbt run --project-dir=/Users/********/development/dbt_projects/sf_sandbox')
r
Raghav Thirumulu
12/15/2020, 8:00 PM@James Parham tried with the full directory. still no luck 😞
h
Hui Zheng
12/15/2020, 8:09 PMthat’s interesting. try to print the error log from DBTshellTask to see why it fails. they recently enabled that . see this issue and the its resolution.
Copy code
<https://github.com/PrefectHQ/prefect/issues/3584>2 Views