Ying Ting Loo
06/12/2024, 2:25 AMMarvin
06/12/2024, 2:26 AMprefect.yaml
deployment file, you need to define the work_pool
and work_queue_name
attributes within the deployments
section. Below is a sample configuration:
yaml
# Generic metadata about this project
name: your-project-name
prefect-version: '2.10.17'
# build section allows you to manage and build docker images
build:
- prefect_docker.deployments.steps.build_docker_image:
id: build_image
requires: prefect-docker>=0.3.0
image_name: your-docker-registry/your-image-name
tag: your-image-tag
dockerfile: Dockerfile
# push section allows you to manage if and how this project is uploaded to remote locations
push:
- prefect_docker.deployments.steps.push_docker_image:
image_name: '{{ build_image.image_name }}'
tag: '{{ build_image.tag }}'
# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect.deployments.steps.git_clone:
repository: your-repo-url
branch: main
access_token: your-access-token
# the deployments section allows you to provide configuration for deploying flows
deployments:
- name: your-deployment-name
version: '{{ build_image.tag }}'
description: Your deployment description
tags: ['your-tag']
schedule: {}
flow_name: your-flow-name
entrypoint: your_flow_file.py:your_flow_function
parameters: {}
work_pool:
name: your-work-pool-name
work_queue_name: your-work-queue-name
job_variables:
image: '{{ build_image.image }}'
# Optionally, define common definitions for reuse
definitions:
common_work_pool: &common_work_pool
name: your-work-pool-name
work_queue_name: your-work-queue-name
job_variables:
image: '{{ build_image.image }}'
# Using common definitions in deployments
deployments:
- name: your-deployment-name
version: '{{ build_image.tag }}'
description: Your deployment description
tags: ['your-tag']
schedule: {}
flow_name: your-flow-name
entrypoint: your_flow_file.py:your_flow_function
parameters: {}
work_pool: *common_work_pool
Make sure you replace placeholders like your-project-name
, your-docker-registry
, your-image-name
, etc., with actual values relevant to your project.
For more details, refer to the links provided:
- Prefect Work Pools Concept
- Prefect Deployment Guide