Mike Dedys
12/11/2025, 1:39 PMMarvin
12/11/2025, 1:39 PMMarvin
12/11/2025, 1:44 PMprefect.yaml
- Reference the image from each deployment’s job variables
# prefect.yaml
name: my-project
build:
- prefect_docker.deployments.steps.build_docker_image:
id: build-image
requires: prefect-docker>=0.3.1
image_name: my-registry/my-image
tag: "{{ prefect.deployments.work_pool_name | default('ci') }}-{{ prefect.deployments.name }}"
dockerfile: ./Dockerfile
push:
- prefect_docker.deployments.steps.push_docker_image:
requires: prefect-docker>=0.3.1
image_name: "{{ build-image.image_name }}"
tag: "{{ build-image.tag }}"
deployments:
- name: ingest
entrypoint: flows/ingest.py:ingest
work_pool:
name: docker-pool
job_variables:
image: "{{ build-image.image }}"
- name: transform
entrypoint: flows/transform.py:transform
work_pool:
name: docker-pool
job_variables:
image: "{{ build-image.image }}"
Then run:
prefect deploy --all
The build/push runs once; both deployments use the same image.
Tip: Use a stable tag like your commit SHA so all deployments in that CI run point to the same image:
- Pre-step to capture git rev-parse --short HEAD
- Set tag: "{{ get_commit.stdout }}"
2) Image is prebuilt by CI (skip Prefect build entirely)
If you build the image in an earlier CI job, skip building in Prefect and just point deployments at the existing tag:
# prefect.yaml
build: null
push: null
deployments:
- name: ingest
entrypoint: flows/ingest.py:ingest
work_pool:
name: docker-pool
job_variables:
image: "my-registry/my-image:${GIT_SHA}"
- name: transform
entrypoint: flows/transform.py:transform
work_pool:
name: docker-pool
job_variables:
image: "my-registry/my-image:${GIT_SHA}"
Or via Python:
from prefect import flow
@flow
def my_flow(): ...
my_flow.deploy(
name="ingest",
work_pool_name="docker-pool",
image="my-registry/my-image:sha-1234",
build=False,
push=False,
)
Notes
- Within a single prefect deploy --all run, the build step executes once; all deployments can reference its output.
- To control Docker layer caching during builds, pass args in the build step (e.g., pull: true, nocache: false, buildargs, platform).
- This works the same whether your worker runs Docker, Kubernetes, ECS, etc.—each uses the same image reference in job_variables.image.
Docs
- prefect.yaml steps and templating
- Create deployments (CLI)
- Deploy via Python API
- Customize job variables
- prefect-docker integration
If you can share how you’re building today (prefect.yaml vs Python API) and which worker type you’re targeting (Docker/Kubernetes/etc.), I can tailor an example to your setup.Mike Dedys
12/11/2025, 2:08 PMMarvin
12/11/2025, 2:08 PMMarvin
12/11/2025, 2:09 PM--name pattern filter, or split configs per env. There isn’t a first-class “group” concept, but these patterns work well in CI.
Options:
1) Name-based groups with --name
- Give each deployment an env-specific name (e.g., dev, staging, prod) and select with wildcards.
Examples:
# deploy all dev across all flows in the project
prefect deploy -n "*/dev"
# deploy only prod deployments (deployment names starting with 'prod-')
prefect deploy -n "*/*-prod*"
# deploy dev for a specific flow only
prefect deploy -n "orders/dev"
Tip: quote patterns so your shell doesn’t expand them.
2) Separate prefect.yaml files per environment
- Keep environment-specific deployments in separate files and call the right one in CI:
prefect deploy --prefect-file prefect.dev.yaml
prefect deploy --prefect-file prefect.staging.yaml
prefect deploy --prefect-file prefect.prod.yaml
3) Programmatic deploy (Python)
- If you prefer code, a tiny script can deploy just one env’s set (handy when your logic gets more complex).
Notes
- The --all flag deploys everything; for env subsets, prefer -n/--name patterns or separate files.
- There’s no CLI filter by tags for creation; selection is by name pattern or file.
Docs
- Create deployments (CLI)
- prefect.yaml structure
If you share how your deployments are currently named, I can suggest exact patterns (e.g., whether you use dev-foo, foo-dev, or dev as the deployment name).