Awesome — here’s a minimal, working repo layout and CI that meets your requirements in Prefect 3.x: - Reusable Docker images in a dedicated folder - A folder of flows where each flow has its own prefect.yaml - A GitHub Action that: - Detects changed images and builds/pushes only those - Detects changed flows and redeploys only those - Redeploys flows that depend on changed images - Points to your external Prefect Server at x.x.x.x Assumptions - Prefect 3.x - You’ll use a Docker work pool (worker runs on your infra with Docker) - You’ll push images to GHCR (GitHub Container Registry) - Your repo is public (if private, see notes for using tokens in pull steps) Links you may want: - prefect.yaml reference - Create deployments - Docker work pools - Self-hosted Prefect Server Repository layout

.
├── images/
│   ├── base-python/
│   │   └── Dockerfile
│   └── data-tools/
│       └── Dockerfile
├── flows/
│   ├── flow-a/
│   │   ├── flow.py
│   │   ├── prefect.yaml
│   │   ├── requirements.txt
│   │   └── image.txt        # contains the image name used by this flow (e.g. "base-python")
│   └── flow-b/
│       ├── flow.py
│       ├── prefect.yaml
│       ├── requirements.txt
│       └── image.txt        # e.g. "data-tools"
└── .github/
    └── workflows/
        └── prefect-ci.yml

images/base-python/Dockerfile

FROM python:3.11-slim

# needed for git_clone pull step
RUN apt-get update && apt-get install -y git && rm -rf /var/lib/apt/lists/*

# Prefect must be available inside the runtime container
RUN pip install --no-cache-dir -U pip setuptools wheel "prefect>=3,<4"

images/data-tools/Dockerfile

FROM python:3.11-slim
RUN apt-get update && apt-get install -y git && rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir -U pip setuptools wheel "prefect>=3,<4" pandas

flows/flow-a/flow.py

from prefect import flow, get_run_logger

@flow
def my_flow_a():
    logger = get_run_logger()
    logger.info("Hello from flow-a!")

if __name__ == "__main__":
    my_flow_a()

flows/flow-a/requirements.txt

# add runtime Python deps for this flow if needed

flows/flow-a/image.txt

base-python

flows/flow-a/prefect.yaml

name: flow-a-project
prefect-version: ">=3.0.0"

deployments:
  - name: flow-a
    entrypoint: flows/flow-a/flow.py:my_flow_a
    work_pool:
      name: docker-pool
      job_variables:
        # Use GHCR; base is passed via CI as env var GHCR_IMAGE_BASE
        image: "{{ $GHCR_IMAGE_BASE }}/base-python:latest"
        env:
          PREFECT_LOGGING_LEVEL: INFO
    pull:
      # pull the repo at run time inside the container
      - prefect.deployments.steps.pull.git_clone:
          repository: "<https://github.com/<owner>/<repo>.git>"
          # If private, use `access_token: "{{ $GITHUB_TOKEN }}"` and pass it via secrets
      - prefect.deployments.steps.pull.set_working_directory:
          directory: "flows/flow-a"
      - prefect.deployments.steps.utility.pip_install_requirements:
          requirements_file: "requirements.txt"

Repeat similarly for flow-b: flows/flow-b/flow.py

from prefect import flow, get_run_logger

@flow
def my_flow_b():
    logger = get_run_logger()
    <http://logger.info|logger.info>("Hello from flow-b!")

if __name__ == "__main__":
    my_flow_b()

flows/flow-b/requirements.txt

# add runtime Python deps for this flow if needed

flows/flow-b/image.txt

data-tools

flows/flow-b/prefect.yaml ``` name: flow-b-project prefect-version: ">=3.0.0" deployments: - name: flow-b entrypoint: flows/flow-b/flow.py:my_flow_b work_pool: name: docker-pool job_variables: image: "{{ $GHCR_IMAGE_BASE }}/data-tools:latest" pull:

- prefect.deployments.steps.pull.git_clone: repository: "https://github.com/<owner>/<repo>.git" - prefect.deployments.steps.pull.set_working_directory: directory: "flows/flow-b" - prefect.deployments.steps.utility.pip_install_requirements: requirements_file: "requirements.txt"

GitHub Actions workflow: .github/workflows/prefect-ci.yml
- Builds/pushes only changed images
- Deploys flows that changed OR that reference changed images
- Points to your self-hosted Prefect Server at x.x.x.x (default API path is /api)

name: Prefect CI on: push: branches: [ "main" ] env: # Point to your self-hosted Prefect Server PREFECT_API_URL: http://x.x.x.x:4200/api # Base path for GHCR images (owner/repo inferred from the event) GHCR_IMAGE_BASE: ghcr.io/${{ github.repository_owner }}/${{ github.event.repository.name }} jobs: build-deploy: runs-on: ubuntu-latest permissions: contents: read packages: write steps: - name: Checkout uses: actions/checkout@v4 with: fetch-depth: 2 - name: Set up Python uses: actions/setup-python@v5 with: python-version: "3.11" - name: Install Prefect (with Docker extras) run: | python -m pip install -U "prefect>=3,<4" "prefect[docker]" - name: Log in to GHCR uses: docker/login-action@v3 with: registry: ghcr.io username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - name: Determine changed images and flows id: changes run: | set -euo pipefail BASE_SHA=$(git rev-parse HEAD^) CHANGED_FILES=$(git diff --name-only "$BASE_SHA" HEAD || true) echo "$CHANGED_FILES" > changed_files.txt echo "Changed files:" cat changed_files.txt # changed image dirs under images/<name>/ CHANGED_IMAGES=$(cat changed_files.txt | grep '^images/' | cut -d/ -f2 | sort -u || true) # changed flow dirs under flows/<name>/ CHANGED_FLOWS=$(cat changed_files.txt | grep '^flows/' | cut -d/ -f2 | sort -u || true) echo "images=$CHANGED_IMAGES" >> $GITHUB_OUTPUT echo "flows=$CHANGED_FLOWS" >> $GITHUB_OUTPUT - name: Build and push changed images if: steps.changes.outputs.images != '' run: | set -euo pipefail for IMG in ${{ steps.changes.outputs.images }}; do echo "Building image: $IMG" docker build -t "${GHCR_IMAGE_BASE}/${IMG}:${GITHUB_SHA}" -t "${GHCR_IMAGE_BASE}/${IMG}:latest" "images/${IMG}" docker push "${GHCR_IMAGE_BASE}/${IMG}:${GITHUB_SHA}" docker push "${GHCR_IMAGE_BASE}/${IMG}:latest" done - name: Compute flows impacted by image changes id: impacted run: | set -euo pipefail CHANGED_IMAGES="${{ steps.changes.outputs.images }}" IMPACTED="" if [ -n "$CHANGED_IMAGES" ]; then for FLOW_DIR in flows/*; do [ -d "$FLOW_DIR" ] || continue FLOW=$(basename "$FLOW_DIR") if [ -f "$FLOW_DIR/image.txt" ]; then IMG_NAME=$(cat "$FLOW_DIR/image.txt" | tr -d '[space]') if echo "$CHANGED_IMAGES" | tr ' ' '\n' | grep -qx "$IMG_NAME"; then IMPACTED="$IMPACTED $FLOW" fi fi done fi echo "impacted_flows=$IMPACTED" >> $GITHUB_OUTPUT - name: Merge flow targets (changed + impacted) id: targets run: | set -euo pipefail CHANGED="${{ steps.changes.outputs.flows }}" IMPACTED="${{ steps.impacted.outputs.impacted_flows }}" TARGETS=$(printf "%s\n%s\n" "$CHANGED" "$IMPACTED" | tr ' ' '\n' | sort -u | tr '\n' ' ' | xargs) echo "targets=$TARGETS" >> $GITHUB_OUTPUT

echo "Target flows: $TARGETS" - name: Deploy targeted flows if: steps.targets.outputs.targets != '' env: PREFECT_API_URL: ${{ env.PREFECT_API_URL }} GHCR_IMAGE_BASE: ${{ env.GHCR_IMAGE_BASE }} run: | set -euo pipefail for FLOW in ${{ steps.targets.outputs.targets }}; do echo "Deploying flow: $FLOW" prefect deploy --no-prompt --prefect-file "flows/${FLOW}/prefect.yaml" done - name: No changes detected if: steps.targets.outputs.targets == '' run: echo "No flows to deploy."

One-time setup on your infra
1) Create a Docker work pool and start a worker (ensure it can reach x.x.x.x and pull from GHCR):

# On the machine that will run flows: export PREFECT_API_URL=http://x.x.x.x:4200/api # Install Prefect + Docker extras python -m pip install -U "prefect>=3,<4" "prefect[docker]" # Create a Docker work pool (run once) prefect work-pool create docker-pool --type docker # (If GHCR images are private) log in once: docker login ghcr.io -u <your-gh-username> -p <a GHCR token with read:packages> # Start a worker that polls the pool prefect worker start --pool docker-pool ``` Notes and options - If your repo is private, update the git_clone step to use an access token: - Add a repo-scoped token to GitHub Actions secrets (e.g., GIT_TOKEN) and in pull steps use:

access_token: "{{ $GIT_TOKEN }}"

- Then pass it in the Deploy step env:

GIT_TOKEN: ${{ secrets.GIT_TOKEN }}

- The PREFECT_API_URL for a self-hosted server is typically http//&lt;host&gt;4200/api. For your IP:

<http://x.x.x.x:4200/api>

. See Server CLI docs. - In Prefect 3.x, we use

prefect deploy

(not

prefect deployment build

) and reference a prefect.yaml per flow. See prefect.yaml. - The Docker images use

:latest

. When an image changes, the CI redeploys flows that depend on it so new runs pull the fresh

latest

tag. Questions for you - Is GHCR okay for your registry, or do you want ECR/GCR/ACR instead? - Will your repo be private? If so, I’ll add the token wiring into the pull steps. - Are you set on Docker for execution, or should we target Kubernetes/ECS? I can adjust job_variables and CI accordingly.