Derek
07/28/2023, 2:40 PMNate
07/28/2023, 6:20 PMDerek
07/31/2023, 2:33 PMNate
07/31/2023, 2:38 PMDerek
07/31/2023, 3:33 PMNate
07/31/2023, 3:54 PMimage
to use
that way, when you change your code, you'll just have to rebuild the image being used by the deployments to include the new code, and not redeploy the agent
ideally (in my general opinion) you have a base image that contains all common deps of your flows, but not the source itself, and then you clone the code just before runtime (that way you dont need to rebuild images just because your code changes) - lmk if you're interested in a pattern like that I can link some resourcesDerek
07/31/2023, 3:55 PMNate
07/31/2023, 4:17 PM# Generic metadata about this project
name: prefect-monorepo
prefect-version: 2.11.1
# build section allows you to manage and build docker images
build: null
# push section allows you to manage if and how this project is uploaded to remote locations
push: null
# pull section allows you to provide instructions for cloning this project in remote locations
pull:
- prefect.deployments.steps.git_clone:
repository: <https://github.com/zzstoatzz/prefect-monorepo>
branch: main
access_token: null
# File for configuring project / deployment build, push and pull steps
definitions:
work_pools:
docker_work_pool: &docker_work_pool # set the image on this work pool in the UI
name: docker-work
work_queue_name: default
# or set it as a job_variable here
schedules:
every_ten_minutes: &every_10_minutes
interval: 600
deployments:
- name: healthcheck-demo
entrypoint: src/demo_project/healthcheck.py:healthcheck
schedule: *every_10_minutes
parameters:
message: Don't panic!
work_pool: *docker_work_pool
you can have a prefect.yaml
like this, which is responsible for defining deployments and mapping the deployments to schedules, infra, other config.
and then I would add a simple github action (or similar) that builds your image for your repo with something like
name: Build and Push Docker Image
on:
push:
branches: [ main ]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }} # or wherever your images are going
password: ${{ secrets.DOCKERHUB_PASSWORD }}
- name: Build and push Docker image
uses: docker/build-push-action@v2
with:
context: .
file: path/to/your/Dockerfile
push: true
tags: username/repository:tag
Mitch
08/17/2023, 8:30 PM