Roman Bedroš
05/18/2026, 6:25 AMMarvin
05/18/2026, 6:25 AMMarvin
05/18/2026, 6:27 AMprofiles.yml, while Prefect resolves the secret values for you.
Short version:
- Put credentials in Prefect Secret blocks
- Reference them in profiles.yml
- Run with PrefectDbtRunner
- Prefect will resolve those references and provide them to dbt via env_var(...)
Docs:
- Prefect dbt runner guide
- profiles.yml templating
- dbt env_var docs
Example `profiles.yml`:
yaml
my_project:
target: dev
outputs:
dev:
type: postgres
host: "{{ prefect.blocks.secret.db-host }}"
user: "{{ prefect.blocks.secret.db-user }}"
password: "{{ prefect.blocks.secret.db-password }}"
dbname: analytics
schema: public
threads: 4
Then in Prefect:
python
from pathlib import Path
from prefect import flow
from prefect_dbt import PrefectDbtRunner, PrefectDbtSettings
@flow
def run_dbt():
settings = PrefectDbtSettings(
project_dir=Path("./my_dbt_project"),
profiles_dir=Path.home() / ".dbt",
)
runner = PrefectDbtRunner(settings=settings)
runner.invoke(["build"])
What happens under the hood:
- Prefect reads profiles.yml
- It resolves {{ prefect.blocks... }} references
- It rewrites them into dbt-style {{ env_var('...') }}
- It sets those env vars for the dbt invocation
So yes: you can use environment variables safely without manually writing a credentials file yourself.
If you prefer, you can also set env vars directly before invoking dbt:
python
import os
from prefect_dbt import PrefectDbtRunner
os.environ["DBT_PASSWORD"] = "secret"
PrefectDbtRunner().invoke(["build"])
and use:
yaml
password: "{{ env_var('DBT_PASSWORD') }}"
But for Prefect-managed runs, Secret blocks are the better pattern.
A couple of notes:
- PrefectDbtRunner is the current API in Prefect 3.x.
- You can control project_dir and profiles_dir through PrefectDbtSettings.
- You do not need to manually manage a special context manager; invoke() handles that internally.
If you want, I can also show a complete example for Snowflake, Postgres, or BigQuery credentials.