<@ULVA73B9P> when deploying prefect to different e...
# ask-marvin
h
@Marvin when deploying prefect to different environments (within the same workspace), is there a way to have a kind of a template prefect.yaml file and supply variables in another file (dev, prod) ? And then deploy to each environment
m
thought for 137.6 seconds
Yes — in Prefect 3, the usual way to do this is either: 1. Use one
prefect.yaml
with placeholders
and inject env-specific values at deploy time, or 2. Use separate
prefect.dev.yaml
/
prefect.prod.yaml
files
and deploy with
--prefect-file
prefect deploy
does support
--prefect-file
, so you can do environment-specific files directly. Good options Option A: One template
prefect.yaml
+ environment variables
Copy code
yaml
deployments:
  - name: my-flow
    entrypoint: flows/my_flow.py:my_flow
    work_pool:
      name: "{{ $WORK_POOL_NAME }}"
      job_variables:
        image: "{{ $IMAGE_NAME }}"
    tags:
      - "{{ $ENVIRONMENT }}"
Then deploy per environment:
Copy code
bash
WORK_POOL_NAME=dev-pool IMAGE_NAME=myrepo/myflow:dev ENVIRONMENT=dev prefect deploy
WORK_POOL_NAME=prod-pool IMAGE_NAME=myrepo/myflow:prod ENVIRONMENT=prod prefect deploy
This is often the simplest pattern for CI/CD. Option B: Separate files per environment
Copy code
bash
prefect deploy --prefect-file prefect.dev.yaml
prefect deploy --prefect-file prefect.prod.yaml
That works well if dev/prod differ in more than just a few values. Option C: Multiple deployments in one file
Copy code
yaml
deployments:
  - name: my-flow-dev
    entrypoint: flows/my_flow.py:my_flow
    work_pool:
      name: dev-pool

  - name: my-flow-prod
    entrypoint: flows/my_flow.py:my_flow
    work_pool:
      name: prod-pool
Then deploy selectively:
Copy code
bash
prefect deploy --name my-flow-dev
prefect deploy --name my-flow-prod
Recommended pattern If you want a "template + values file" workflow, Prefect doesn't have a built-in Helm-style
values.yaml
merge system for
prefect.yaml
. So the common approaches are: - Single templated
prefect.yaml
using
{{ $ENV_VAR }}
placeholders - A small wrapper script that loads
.env.dev
or
.env.prod
- or multiple `prefect..yaml` files* with
--prefect-file
For example:
Copy code
bash
set -a
source .env.dev
set +a
prefect deploy
or
Copy code
bash
set -a
source .env.prod
set +a
prefect deploy
If I were setting this up, I’d usually choose: - env vars + one
prefect.yaml
if the differences are small - separate prefect files if the deployment definitions differ materially Docs / references - How to define deployments with prefect.yaml - Deploy flows with Python - Deployments overview If you want, I can sketch a concrete repo layout for `dev`/`prod` with: - one shared
prefect.yaml
-
.env.dev
/
.env.prod
- and the exact deploy commands.
h
@Marvin yes sketch a concrete repo layout for `dev`/`prod` with: - one shared
prefect.yaml
-
.<http://env.dev|env.dev>
/
.env.prod
- and the exact deploy commands.
m
thought for 18.5 seconds
Absolutely — here’s a clean pattern that works well. Example repo layout
Copy code
text
my-project/
├── flows/
│   └── my_flow.py
├── prefect.yaml
├── .env.dev
├── .env.prod
└── deploy.sh
Example flow
Copy code
python
from prefect import flow

@flow
def my_flow():
    print("hello from prefect")

if __name__ == "__main__":
    my_flow()
Shared
prefect.yaml
Copy code
yaml
name: my-project

deployments:
  - name: my-flow
    entrypoint: flows/my_flow.py:my_flow
    work_pool:
      name: "{{ $WORK_POOL_NAME }}"
      job_variables:
        image: "{{ $IMAGE_NAME }}"
        env:
          APP_ENV: "{{ $APP_ENV }}"
    tags:
      - "{{ $APP_ENV }}"
    description: "Deployment for {{ $APP_ENV }}"
That gives you one deployment definition whose values change based on the environment variables you load before running
prefect deploy
.
.env.dev
Copy code
bash
WORK_POOL_NAME=dev-pool
IMAGE_NAME=my-registry/my-project:dev
APP_ENV=dev
.env.prod
Copy code
bash
WORK_POOL_NAME=prod-pool
IMAGE_NAME=my-registry/my-project:prod
APP_ENV=prod
Exact deploy commands For dev:
Copy code
bash
set -a
source .env.dev
set +a
prefect deploy --name my-flow
For prod:
Copy code
bash
set -a
source .env.prod
set +a
prefect deploy --name my-flow
If you want it as one-liners:
Copy code
bash
set -a && source .env.dev && set +a && prefect deploy --name my-flow
set -a && source .env.prod && set +a && prefect deploy --name my-flow
Optional helper script You can make this a little easier with `deploy.sh`:
Copy code
bash
#!/usr/bin/env bash
set -euo pipefail

ENV_NAME="${1:-dev}"

if [[ "$ENV_NAME" == "dev" ]]; then
  set -a
  source .env.dev
  set +a
elif [[ "$ENV_NAME" == "prod" ]]; then
  set -a
  source .env.prod
  set +a
else
  echo "Usage: ./deploy.sh [dev|prod]"
  exit 1
fi

prefect deploy --name my-flow
Then run:
Copy code
bash
./deploy.sh dev
./deploy.sh prod
Important note about deployment names If you deploy both dev and prod with the same deployment name
my-flow
, you may end up overwriting the same deployment config rather than keeping two distinct deployments. If you want both environments to exist side-by-side in the same workspace, make the deployment name environment-specific. For example:
Copy code
yaml
name: my-project

deployments:
  - name: "my-flow-{{ $APP_ENV }}"
    entrypoint: flows/my_flow.py:my_flow
    work_pool:
      name: "{{ $WORK_POOL_NAME }}"
      job_variables:
        image: "{{ $IMAGE_NAME }}"
        env:
          APP_ENV: "{{ $APP_ENV }}"
    tags:
      - "{{ $APP_ENV }}"
Then:
Copy code
bash
set -a && source .env.dev && set +a && prefect deploy --name my-flow-dev
set -a && source .env.prod && set +a && prefect deploy --name my-flow-prod
That is usually the better setup for same workspace, multiple environments. Recommended final version I’d recommend this exact `prefect.yaml`:
Copy code
yaml
name: my-project

deployments:
  - name: "my-flow-{{ $APP_ENV }}"
    entrypoint: flows/my_flow.py:my_flow
    description: "Deployment for {{ $APP_ENV }}"
    tags:
      - "{{ $APP_ENV }}"
    work_pool:
      name: "{{ $WORK_POOL_NAME }}"
      job_variables:
        image: "{{ $IMAGE_NAME }}"
        env:
          APP_ENV: "{{ $APP_ENV }}"
Then deploy with:
Copy code
bash
set -a && source .env.dev && set +a && prefect deploy --name my-flow-dev
set -a && source .env.prod && set +a && prefect deploy --name my-flow-prod
Docs - How to define deployments with prefect.yaml - Deployments overview If you want, I can also show a version that uses Docker build/push steps in the same shared
prefect.yaml
for dev/prod images.