Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi there - I am running into an issue with the `DbtShellTask` ; locally works all as expected. when i deploy to prefect cloud, I am getting a `dbt: command not found` .. requirements include `dbt-redshift` . My flow storage is Docker. when i try to build the Docker image locally, inside the container I also get `dbt: command not found`  even though it is installed in my python environment packages. Has anyone run into this before?

for context:
```dbt_run = DbtShellTask(
    environment=ENVIRONMENT,
    profile_name="redshift",
    dbt_kwargs={
        "type": "redshift",
        "host": REDSHIFT_HOST,
        "user": REDSHIFT_USER,
        "password": REDSHIFT_PASSWORD,
        "port": REDSHIFT_PORT,
        "database": REDSHIFT_DATABASE,
        "schema": REDSHIFT_SCHEMA,
        "threads": 4,
        "client_session_keep_alive": False,
    },
    helper_script=f"cd {GIT_REPOSITORY}/redshift",
    overwrite_profiles=True,
    return_all=True,
    log_stdout=True,
    log_stderr=True,
)```

and then within flow:
```with Flow(name="dbt-xxx") as flow:
    repo = clone_repo()
    logs = dbt_run(
        task_args={"name": "xxx"},
        command="dbt run --models xxx", upstream_tasks=[repo]
    )```


Forgot to install `dbt-redshift` package in the container? You can verify by running `pip list` in a shell in the container…

```# pip list | grep dbt
dbt-core                    1.1.0
dbt-extractor               0.4.1
dbt-postgres                1.1.0
dbt-redshift                1.1.0```

Might be a PYTHONPATH issue, also if you run your docker container with sudo, it changes the directory and this may cause issues - worth checking your user in the container

even if i do something like `/usr/local/bin/python -m dbt` still command not found

Running that command gives me:
```/usr/local/bin/python: No module named dbt.__main__; 'dbt' is a package and cannot be directly executed```


Doing some more digging we checked for active commands that we could run using the following:
```compgen -ac | grep dbt```
Which returned nothing for dbt

Then we uninstalled all dbt related packages:
```pip uninstall dbt-core dbt-extractor dbt-postgres dbt-redshift```
After a reinstall
```pip install dbt-core dbt-redshift```


Afterwards, this shows dbt is available
```compgen -ac | grep dbt```
And `dbt --help`  works, I think it might have to do with our Docker image

Strange… is it possible to share your Dockerfile?

I think we figured it out, we use a builder step in our dockerfile and copy site packages to final step, which was wiping out the CLI, after moving the dbt install in the final step its seems that the dbt CLI is available