Javier SG
06/30/2025, 9:49 AMname: backend-flows
prefect-version: 3.4.7
build: null
push: null
pull:
- prefect.deployments.steps.git_clone:
repository: <https://github.com/repo/backend.git>
branch: main
access_token: "{{ prefect.blocks.secret.github-access-token }}"
deployments:
- name: process
tags: ["main"]
entrypoint: flows/subfolder/process.py:lmain
parameters: {}
work_pool:
name: my-workpool
job_variables:
image: my/docker/image:latest
Marvin
06/30/2025, 9:49 AMMarvin
06/30/2025, 9:51 AMprefect.yaml
file that incorporates a Docker build target. While Prefect doesn't directly expose the target flag in its configuration, you can achieve this using the build_config
option. Here's an example:
yaml
prefect.yaml
---
name: my-deployment
version: 0.1.0
build:
- prefect_docker.deployments.steps.build_docker_image:
requires: prefect-docker>=0.3.1
image_name: my-flow-image
tag: latest
dockerfile: Dockerfile
build_config:
buildargs: {}
target: prefect # This specifies the build target
platform: linux/amd64
push: false # Set to true if you want to push to a registry
deployments:
- name: example-deployment
version: 0.1.0
tags: []
description: "Example deployment using Docker with a specific target"
entrypoint: flow.py:my_flow
work_pool:
name: my-docker-pool
work_queue_name: default
For this to work, your Dockerfile should use multi-stage builds with the appropriate target. Here's a complementary example of what your Dockerfile might look like:
dockerfile
# Base stage
FROM python:3.9-slim as base
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
# Development stage
FROM base as development
COPY dev-requirements.txt .
RUN pip install -r dev-requirements.txt
# Prefect stage (this is our target)
FROM base as prefect
COPY . .
RUN pip install prefect
A few important notes:
1. The target: prefect
in the build_config
corresponds to the as prefect
stage in the Dockerfile
2. Make sure you have the prefect-docker
package installed (pip install prefect-docker>=0.3.1
)
3. The build_config
section allows you to pass various Docker build arguments, including the target
4. The example assumes you're using a Docker work pool for execution
You can deploy this configuration using:
bash
prefect deploy
This setup allows you to maintain different build stages in your Dockerfile while specifically using the prefect
target for your Prefect deployments.
Links for further reading:
- Prefect Docker Deployments Documentation
- Prefect.yaml Configuration GuideMarvin
06/30/2025, 10:01 AMMarvin
06/30/2025, 10:01 AMprefect.yaml
to use a Docker image with the --prefect
target:
yaml
name: backend-flows
prefect-version: 3.4.7
build:
- prefect_docker.deployments.steps.build_docker_image:
requires: prefect-docker>=0.3.1
image_name: my/docker/image
tag: latest
dockerfile: Dockerfile
build_config:
target: prefect # This specifies the build target
push: null
pull:
- prefect.deployments.steps.git_clone:
repository: <https://github.com/repo/backend.git>
branch: main
access_token: "{{ prefect.blocks.secret.github-access-token }}"
deployments:
- name: process
tags: ["main"]
entrypoint: flows/subfolder/process.py:lmain
parameters: {}
work_pool:
name: my-workpool
job_variables:
image: my/docker/image:latest
The key changes I made:
1. Added the build
section with prefect_docker.deployments.steps.build_docker_image
2. Specified the target as prefect
in the build_config
3. Kept your existing pull
configuration for Git
4. Maintained your deployment configuration
Make sure your Dockerfile has the corresponding target defined, like this:
dockerfile
# Base stage
FROM python:3.9-slim as base
# ... base setup ...
# Target stage for Prefect
FROM base as prefect
# ... prefect-specific setup ...
To use this configuration:
1. Install the required package: pip install prefect-docker>=0.3.1
2. Make sure your Dockerfile is in the same directory as your prefect.yaml
(or adjust the path in the dockerfile
field)
3. Deploy using prefect deploy
The Docker image will be built using the prefect
target before the deployment is created.Javier SG
06/30/2025, 10:02 AMMarvin
06/30/2025, 10:02 AMMarvin
06/30/2025, 10:02 AMjob_variables
section of your work pool configuration. Here's how to modify your `prefect.yaml`:
yaml
name: backend-flows
prefect-version: 3.4.7
build: null
push: null
pull:
- prefect.deployments.steps.git_clone:
repository: <https://github.com/repo/backend.git>
branch: main
access_token: "{{ prefect.blocks.secret.github-access-token }}"
deployments:
- name: process
tags: ["main"]
entrypoint: flows/subfolder/process.py:lmain
parameters: {}
work_pool:
name: my-workpool
job_variables:
image: my/docker/image:latest
target: prefect # Specifies which target to use when running the container
This configuration:
1. Maintains your existing setup where the image is built externally in GitHub Actions
2. Adds the target: prefect
to the job_variables
to specify which target from your multi-stage Dockerfile should be used
3. Keeps all your other configurations intact
This assumes your Docker image is built with multi-stage builds and has a target named prefect
defined in its Dockerfile.