Fabio Machado
09/17/2024, 6:57 PMMarvin
09/17/2024, 6:58 PMprefect.yaml
in CI/CD pipelines, here are insights from the latest Prefect documentation:
### Prefect 2.x
1. Overview:
- The prefect.yaml
file is used to store and manage configuration for deploying flows.
- This file is typically committed to source control along with your flow code.
2. Structure:
- The file includes metadata about the project, build configurations, push steps, and pull configurations.
- It commonly integrates Docker steps for building and pushing Docker images.
3. Example Configuration:
You can use a basic setup like the following in your `prefect.yaml`:
yaml
name: <YOUR_PROJECT_NAME>
prefect-version: 2.0.0
build:
- prefect_docker.deployments.steps.build_docker_image:
id: build_image
requires: prefect-docker>=0.3.1
image_name: <YOUR_IMAGE_NAME>
tag: latest
dockerfile: auto
platform: linux/amd64
push:
- prefect_docker.deployments.steps.push_docker_image:
requires: prefect-docker>=0.3
This setup builds and pushes Docker images as part of the CI/CD process.
### Prefect 3.x
1. Best Practices:
- Prefect 3.x continues to support the prefect.yaml
file, with extended capabilities for CI/CD pipeline integration.
- It emphasizes handling multiple workspaces and project folders within a monorepo.
2. Example Repository Structure:
- Use CI/CD workflows to deploy to multiple workspaces.
- Organize projects to have their own flows, dependencies, and configuration files (including prefect.yaml
).
For example:
.
├─ cicd-example-workspaces-prod
│ ├─ project_1
│ │ ├─ prefect.yaml
│ │ └─ ...
│ └─ ...
3. Workflow Configuration:
- Each project folder can have a prefect.yaml
file with environment-specific configurations.
- GitHub Actions or other CI/CD tools can trigger deployments based on branch conditions and changes in project folders.
For more detailed examples and advanced use-cases, please refer to the official documentation:
- Prefect 2.x CI/CD Guide: Prefect 2.x CI/CD Guide
- Prefect 3.x Infrastructure and CI/CD Concepts: Prefect 3.x CI/CD Guide
Feel free to copy these examples into your projects and adapt them as needed!