<@ULVA73B9P> I deploy my flows to Prefect OSS on K...
# ask-marvin
k
@Marvin I deploy my flows to Prefect OSS on Kubernetes from CI using prefect.yaml and github workflows. I want to pass the container image in
job_variables
, but for multiple flows it doesn't work. Is there a way to do it?
m
thought for 219.0 seconds
Short answer: yes—you can set a different container image per deployment with
job_variables
in Prefect 3.x. The key is to scope the image value per deployment. If you template a single value at the top (or use one build step output for all), every deployment will get the same image. Here are working patterns you can use from CI: Pattern A: Per-deployment env var interpolation - Put the image directly in each deployment’s
work_pool.job_variables.image
, and populate different env vars for each deployment from CI. prefect.yaml
Copy code
deployments:
  - name: flow-a
    entrypoint: flows/flow_a.py:flow
    work_pool:
      name: my-k8s
      job_variables:
        image: "{{ $FLOW_A_IMAGE }}"
        namespace: "prefect"

  - name: flow-b
    entrypoint: flows/flow_b.py:flow
    work_pool:
      name: my-k8s
      job_variables:
        image: "{{ $FLOW_B_IMAGE }}"
        namespace: "prefect"
GitHub Actions (two steps, one env var per deploy)
Copy code
- name: Deploy flow-a
  run: |
    export FLOW_A_IMAGE=<http://ghcr.io/org/flow-a:${GITHUB_SHA}|ghcr.io/org/flow-a:${GITHUB_SHA}>
    prefect deploy --name flow-a

- name: Deploy flow-b
  run: |
    export FLOW_B_IMAGE=<http://ghcr.io/org/flow-b:${GITHUB_SHA}|ghcr.io/org/flow-b:${GITHUB_SHA}>
    prefect deploy --name flow-b
Tip: If you run a single
prefect deploy --all
, any env vars you set will be shared across deployments—so use separate env vars (and/or deploy each by name) when you want different images. Pattern B: Deployment-scoped image build steps - Define a build step per deployment and reference its output in that deployment’s
job_variables.image
. This lets you run
prefect deploy --all
and still get different images. prefect.yaml
Copy code
deployments:
  - name: flow-a
    entrypoint: flows/flow_a.py:flow
    build:
      - prefect_docker.deployments.steps.build_docker_image:
          id: build-image
          requires: "prefect-docker>=0.4.0"
          image_name: "<http://ghcr.io/org/flow-a|ghcr.io/org/flow-a>"
          tag: "{{ $GITHUB_SHA }}"
    work_pool:
      name: my-k8s
      job_variables:
        image: "{{ build-image.image }}"
        namespace: "prefect"

  - name: flow-b
    entrypoint: flows/flow_b.py:flow
    build:
      - prefect_docker.deployments.steps.build_docker_image:
          id: build-image
          requires: "prefect-docker>=0.4.0"
          image_name: "<http://ghcr.io/org/flow-b|ghcr.io/org/flow-b>"
          tag: "{{ $GITHUB_SHA }}"
    work_pool:
      name: my-k8s
      job_variables:
        image: "{{ build-image.image }}"
        namespace: "prefect"
In CI, make sure the Docker integration is available when deploying:
Copy code
uv run --with 'prefect[docker]' prefect deploy --all
Common gotchas - One build step id for many deployments: If you have a single top-level build step like
id: build-image
and you reference
{{ build-image.image }}
in multiple deployments, they’ll all get the same image. Use per-deployment build steps or per-deployment env vars instead. - One env var for many deployments: A single
IMAGE
env var will be the same for all deployments in that process. Use
FLOW_A_IMAGE
,
FLOW_B_IMAGE
, etc., or run
prefect deploy --name <deployment>
multiple times with different env vars for each. If you share a snippet of your current
prefect.yaml
and how you invoke
prefect deploy
in GitHub Actions, I can pinpoint exactly why it’s not working and tailor the fix. Helpful docs: - Prefect YAML reference: Define deployments with prefect.yaml - Customize job variables: Customize job variables - Kubernetes worker guide: Run flows on Kubernetes