Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hey Everyone - For those using <https://docs.prefect.io/api/latest/tasks/dbt.html#dbtshelltask|DbtShellTask>, is it expected to have to run `dbt compile` before executing `dbt run`?

Hi <@U02DS7BMTLK> it’s the first time I hear about dbt compile :smile: I didn’t have to use it. Here is a demo in a blog post: <https://www.prefect.io/blog/flow-of-flows-orchestrating-elt-with-prefect-and-dbt/>

Thanks <@U02H1A95XDW>! I’ll take a look

For some more context my flow looks like this
```with Flow(FLOW_NAME) as flow:
    pull_repo = pull_from_git(repository=github_task.task_config['repository'])
    deps = dbt(
        command="dbt deps",
        upstream_tasks=[pull_repo],
    )
    run = dbt(
        command='dbt run -m +&lt;MODEL_NAME&gt;',
        upstream_tasks=[pull_repo, deps],
    )

if __name__ == '__main__':
    flow.run()```

If I just run this I get a model specification error saying that the dbt node &lt;MODEL_NAME&gt; doesn’t exist

but if I change the first command to `command="dbt deps &amp; dbt compile"`

on the dbt shell task:
```helper_script=f"cd {DBT_PROJECT}",```

Yep, my task looks like this
```dbt = DbtShellTask(
    return_all=True,
    profile_name='&lt;PROFILE_NAME&gt;',
    environment='prod',
    set_profiles_envar=False,
    overwrite_profiles=False,
    log_stdout=True,
    helper_script='cd dbt',
    log_stderr=True,
    dbt_kwargs={&lt;DICT OF SECRETS&gt;
    },```


interesting. I always use:
```overwrite_profiles=True,```
this ensures that a new clean dbt profile gets created based on the kwargs set on this task. And since I saw you clone the repository anyway, it should do no harm to overwrite it. Maybe you can give it a try.

it’s a demo effect :smile: nice that it’s working now!

I don’t think `overwrite_profiles=True,` will work for us because we’re an agency and work across 10+ dbt/prefect instances

so overwriting profiles.yml would remove all of the other creds too

But seems to be working now, thanks your help!