https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P> how to make some deployment to use di...
# ask-marvin
f
@Marvin how to make some deployment to use differenct docker file. # Welcome to your prefect.yaml file! You can use this file for storing and managing # configuration for deploying your flows. We recommend committing this file to source # control along with your flow code. # Generic metadata about this project name: prefect prefect-version: 3.4.7 # build section allows you to manage and build docker images build: null # - prefect_docker.deployments.steps.build_docker_image: # id: build_image # requires: prefect-docker>=0.3.1 # image_name: fugotakefusaptml/prefect-docker-repo # tag: 2 # dockerfile: auto # push section allows you to manage if and how this project is uploaded to remote locations push: null # - prefect_docker.deployments.steps.push_docker_image: # requires: prefect-docker>=0.3.1 # image_name: '{{ build_image.image_name }}' # tag: '{{ build_image.tag }}' # pull section allows you to provide instructions for cloning this project in remote locations pull: null # - prefect.deployments.steps.set_working_directory: # directory: /opt/prefect/prefect definitions: work_pools: docker_worker_pool: &docker_worker_pool name: docker-worker-pool work_queue_name: null job_variables: image: '{{ build_image.image }}' actions: docker_build: &docker_build - prefect_docker.deployments.steps.build_docker_image: &docker_build_config id: build_image requires: prefect-docker>=0.3.0 image_name: fugotakefusaptml/prefect-docker-repo tag: dev dockerfile: "sample.Dockerfile" docker_push: &docker_push - prefect_docker.deployments.steps.push_docker_image: &docker_push_config requires: prefect-docker>=0.3.1 image_name: fugotakefusaptml/prefect-docker-repo tag: dev # the deployments section allows you to provide configuration for deploying flows deployments: - name: deploy_buy1 flow_name: deploybuy entrypoint: data_flows/deploy_buy.py:buy work_pool: *docker_worker_pool build: *docker_build # push: *docker_push
m
✅ thought for 171.6 seconds
You can give each deployment its own Dockerfile by either: - defining multiple build “actions” (anchors) in prefect.yaml and referencing the one you want per deployment, or - inlining the build step in the deployment and setting a different 
dockerfile
. Key points: - Use 
prefect_docker.deployments.steps.build_docker_image
with 
dockerfile: "<path-to-your-Dockerfile>"
. - Reference the built image in your work pool 
job_variables.image
with 
{{ build_image.image }}
(the step’s 
id
must be 
build_image
or you must adjust the template accordingly). - Keep tags unique if you’re pushing to the same repo to avoid collisions. Option A: multiple build anchors with different Dockerfiles
Copy code
name: prefect
prefect-version: 3.4.7

definitions:
  work_pools:
    docker_worker_pool: &docker_worker_pool
      name: docker-worker-pool
      work_queue_name: null
      job_variables:
        image: '{{ build_image.image }}'

  actions:
    # Base build config (to merge into others)
    docker_build_base: &docker_build_base
      - prefect_docker.deployments.steps.build_docker_image: &docker_build_config
          id: build_image
          requires: prefect-docker>=0.3.1
          image_name: fugotakefusaptml/prefect-docker-repo
          tag: dev
          dockerfile: Dockerfile

    # Variant 1: uses sample.Dockerfile
    docker_build_sample: &docker_build_sample
      - prefect_docker.deployments.steps.build_docker_image:
          <<: *docker_build_config
          tag: sample
          dockerfile: sample.Dockerfile

    # Variant 2: uses buy.Dockerfile
    docker_build_buy: &docker_build_buy
      - prefect_docker.deployments.steps.build_docker_image:
          <<: *docker_build_config
          tag: buy
          dockerfile: buy.Dockerfile

    # Variant 3: uses sell.Dockerfile
    docker_build_sell: &docker_build_sell
      - prefect_docker.deployments.steps.build_docker_image:
          <<: *docker_build_config
          tag: sell
          dockerfile: sell.Dockerfile

deployments:
  - name: deploy_buy1
    flow_name: deploybuy
    entrypoint: data_flows/deploy_buy.py:buy
    work_pool: *docker_worker_pool
    build: *docker_build_buy

  - name: deploy_sell1
    flow_name: deploysell
    entrypoint: data_flows/deploy_sell.py:sell
    work_pool: *docker_worker_pool
    build: *docker_build_sell
Option B: inline per-deployment build steps
Copy code
deployments:
  - name: deploy_buy1
    flow_name: deploybuy
    entrypoint: data_flows/deploy_buy.py:buy
    work_pool:
      name: docker-worker-pool
      job_variables:
        image: '{{ build_image.image }}'
    build:
      - prefect_docker.deployments.steps.build_docker_image:
          id: build_image
          requires: prefect-docker>=0.3.1
          image_name: fugotakefusaptml/prefect-docker-repo
          tag: buy
          dockerfile: buy.Dockerfile

  - name: deploy_sample1
    flow_name: deploysample
    entrypoint: data_flows/deploy_sample.py:sample
    work_pool:
      name: docker-worker-pool
      job_variables:
        image: '{{ build_image.image }}'
    build:
      - prefect_docker.deployments.steps.build_docker_image:
          id: build_image
          requires: prefect-docker>=0.3.1
          image_name: fugotakefusaptml/prefect-docker-repo
          tag: sample
          dockerfile: sample.Dockerfile
Notes: - Using the same 
id: build_image
in each deployment is fine; each deployment resolves its own step results. - If you push images, add corresponding push steps (anchored or inline) and keep 
tag
consistent. Docs and references: - Prefect 3 prefect.yaml guide: prefect.yaml (3.x) - Docker steps source (shows available parameters): prefect_docker.deployments.steps
If you share your exact set of Dockerfiles and which deployments should use each, I can tailor the YAML for you.
Open in Slack
PreviousNext
Powered by