Short answer: dbt is complaining because

--profiles-dir

is being parsed as an option for the subcommand instead of as a global option. This usually happens when the flag is passed in the wrong place (after the subcommand) or when it’s being added by Prefect in a way dbt doesn’t expect. Common causes and fixes: - You’re passing

--profiles-dir

in the

invoke([...])

args. With PrefectDbtRunner, don’t pass global dbt flags in the args; configure them via settings or env vars instead. - Correct approach:

runner = PrefectDbtRunner(
      settings=PrefectDbtSettings(
          project_dir="/path/to/project",
          profiles_dir="/path/to/profiles"
      )
  )
  runner.invoke(["build"])

- Or set env vars:

export DBT_PROJECT_DIR=/path/to/project
  export DBT_PROFILES_DIR=/path/to/profiles

- If you must use CLI-style args, dbt’s global options must come before the subcommand. For example,

dbt --profiles-dir /path build

is valid, but

dbt build --profiles-dir /path

can be rejected in contexts where Click is parsing options for the subcommand. This is especially problematic for nested commands like

source freshness

(known ordering issue). - Related issue (older CLI wrapper, but same root cause): prefect#13612 Additional context: - There are a couple of recent threads about how Prefect resolves

profiles_dir

and defaults to

~/.dbt

if not set, which can surprise users in containers (e.g.,

/root/.dbt

not found). If you see that instead, set

DBT_PROFILES_DIR

or

PrefectDbtSettings(profiles_dir=...)

. - Path resolution discussion: prefect#17555 - Missing

/root/.dbt

example: prefect#18412 - Prefect 3.x docs for prefect-dbt: Prefect dbt integration If you can share: - Your

PrefectDbtRunner

invocation (the exact

invoke([...])

args) - Versions of

prefect

,

prefect-dbt

, and

dbt-core

I can pinpoint whether it’s strictly argument ordering or a settings resolution edge case.

@Marvin, can you pinpoint whether it’s strictly argument ordering or a settings resolution edge case?

Short answer: based on the exact error text “No such option: --profiles-dir” for the subcommand ‘build’, this is almost certainly argument ordering, not a settings-resolution edge case. Why I think that: - When profiles path resolution fails, dbt errors look like “Path '/.../.dbt' does not exist.” (example: prefect#18412). - “No such option: --profiles-dir” happens when Click is parsing flags for the subcommand instead of global flags—i.e.,

--profiles-dir

is placed after the subcommand (or injected there by a wrapper). This is a known class of issue with dbt CLI flag ordering (see older wrapper example: prefect#13612). How to confirm in your env: 1) Remove any CLI flags and rely on settings:

runner = PrefectDbtRunner(
  settings=PrefectDbtSettings(
    project_dir="/path/to/project",
    profiles_dir="/path/to/profiles",
  )
)
runner.invoke(["build"])

If this works, the prior failure was flag ordering. 2) If you must pass CLI-style args, put global flags before the subcommand:

runner.invoke(["--profiles-dir", "/path/to/profiles", "build"])

If this works but

["build", "--profiles-dir", ...]

fails, it’s 100% ordering. If you share: - Your exact

runner.invoke([...])

args, - Whether you set

PrefectDbtSettings(profiles_dir=...)

or

DBT_PROFILES_DIR

, - Versions of

prefect

,

prefect-dbt

, and

dbt-core

, I can confirm definitively and tailor a fix.

Great question. Two important points first: - PrefectDbtRunner does not spawn a shell process; it uses dbt’s Python API. So there isn’t a single shell “command” string. Instead, it builds an args list and passes it to dbt’s runner. - You can still see what it’s about to invoke by enabling debug logs for the prefect-dbt logger. How to see the constructed args Option A: Turn on DEBUG logging for prefect-dbt - Set an env var before your run:

PREFECT_LOGGING_LEVEL=DEBUG

- Or configure the logger in your flow/task:

import logging
from prefect_dbt.core.runner import PrefectDbtRunner
from prefect_dbt.core.settings import PrefectDbtSettings

# Ensure debug output from the integration
logging.getLogger("prefect_dbt").setLevel(logging.DEBUG)
logging.getLogger("prefect_dbt.core.runner").setLevel(logging.DEBUG)

runner = PrefectDbtRunner(
    settings=PrefectDbtSettings(
        project_dir="/path/to/project",
        profiles_dir="/path/to/profiles",
    )
)
runner.invoke(["build"])

With DEBUG enabled, prefect-dbt emits messages showing the args it passes to dbt (including where

--profiles-dir

ends up). Option B: Add a temporary logging handler to guarantee visibility

import logging
handler = logging.StreamHandler()
handler.setLevel(logging.DEBUG)
for name in ["prefect_dbt", "prefect_dbt.core.runner"]:
    lg = logging.getLogger(name)
    lg.setLevel(logging.DEBUG)
    lg.addHandler(handler)

Option C (diagnostic alternative): Use DbtCoreOperation just to inspect the CLI string DbtCoreOperation prints the exact shell command it runs (nice for debugging ordering), even if you plan to use PrefectDbtRunner long-term.

from prefect_dbt.cli.commands import DbtCoreOperation

DbtCoreOperation(
    commands=["dbt build"],
    project_dir="/path/to/project",
    profiles_dir="/path/to/profiles",
).run()

You’ll see the concrete CLI it executes in the logs. If you enable DEBUG and don’t see the args, let me know your versions of prefect, prefect-dbt, and dbt-core; I can tailor the exact logger names or provide a tiny monkeypatch to print the final args. For reference: prefect-dbt docs.

Thanks for flagging this — I haven’t found an existing report of that specific regression in prefect-dbt 0.7.7, so let’s confirm and unblock you. Quick ways to prove the stripping bug and capture evidence - Enable DEBUG logs for prefect-dbt so we can see exactly what args PrefectDbtRunner is building and handing to dbt:

import logging
logging.getLogger("prefect_dbt").setLevel(logging.DEBUG)
logging.getLogger("prefect_dbt.core.runner").setLevel(logging.DEBUG)

Then run your flow and share the log lines that show the built args. - Minimal subclass to print what invoke receives:

from prefect_dbt.core.runner import PrefectDbtRunner

class DebugRunner(PrefectDbtRunner):
    def invoke(self, args, **kwargs):
        print(f"invoke received args={args}")
        return super().invoke(args, **kwargs)

runner = DebugRunner(...)
runner.invoke(["build"])

If the print shows

["build"]

but the logs show the command missing later, the stripping happens inside the runner; if it’s already missing at entry, something higher up is altering it. Immediate workarounds - Prefer settings/env over CLI-style args so no command parsing is required:

from prefect_dbt.core.runner import PrefectDbtRunner
from prefect_dbt.core.settings import PrefectDbtSettings

runner = PrefectDbtRunner(
  settings=PrefectDbtSettings(
    project_dir="/path/to/project",
    profiles_dir="/path/to/profiles"
  )
)
runner.invoke(["build"])

- If you need a concrete CLI for now, switch to the CLI-based operator temporarily:

from prefect_dbt.cli.commands import DbtCoreOperation

DbtCoreOperation(
  commands=["dbt build"],
  project_dir="/path/to/project",
  profiles_dir="/path/to/profiles",
).run()

Next steps - If you can share: - The exact

runner = PrefectDbtRunner(...)

setup and

runner.invoke([...])

line - Your versions of

prefect

,

prefect-dbt

(0.7.7), and

dbt-core

- The DEBUG logs showing the constructed args I’ll verify precisely where the command is being dropped and open a bug with a minimal repro (or you can file it directly here: Open a Prefect bug report). Also, if you can try either upgrading to the latest

prefect-dbt

or downgrading to 0.7.6, that will tell us whether it’s isolated to 0.7.7.