Great question. Short answer: - Run long‑lived Prefect worker(s) in your Docker Compose stack on the Oracle VM - Create/update deployments from CI/CD (often via a temporary container that runs

prefect deploy

or a small Python

flow.deploy()

script) - Let the worker pick up scheduled runs from a work pool and execute them (typically as ephemeral Docker containers) Below is a concise pattern that works well in Prefect 3.x. Best-practice architecture - Work pool + worker(s) in Compose: - Docker work pool = each flow run executes in an ephemeral container (recommended) - Process work pool = runs as subprocesses inside the worker container (simpler, less isolation) - Deployments created by CI: - Either with

prefect deploy

and a

prefect.yaml

- Or with a small Python script using

flow.deploy()

or

flow.from_source(...).deploy(...)

One-time setup - Decide where the API lives: - Prefect Cloud: set

PREFECT_API_URL

and

PREFECT_API_KEY

on the worker(s) - Self‑hosted: run Prefect Server in Compose and point worker(s) to it - Create a work pool (once):

prefect work-pool create --type docker docker-pool

Docker Compose: worker service (Docker work pool)

services:
  prefect-worker-docker:
    image: prefecthq/prefect:3-latest
    environment:
      # For Prefect Cloud:
      # PREFECT_API_URL: <https://api.prefect.cloud/api/accounts/${PREFECT_ACCOUNT_ID}/workspaces/${PREFECT_WORKSPACE_ID}>
      # PREFECT_API_KEY: ${PREFECT_API_KEY}
      # Or self-hosted:
      # PREFECT_API_URL: <http://prefect-server:4200/api>
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock  # allow worker to launch flow run containers
    command: >
      prefect worker start
      --pool docker-pool
      --name docker-worker-1
    restart: unless-stopped

Deployments from CI/CD - Yes, use a temporary container to run deployment commands in CI. It gives you a clean, reproducible environment and matches the runtime image. Option A:

prefect.yaml

+ CLI - Commit a

prefect.yaml

and run

prefect deploy

in CI. GitHub Actions example:

name: Deploy Prefect
on:
  push:
    branches: [main]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Deploy via temporary container
        run: |
          docker run --rm \
            -e PREFECT_API_URL=${{ secrets.PREFECT_API_URL }} \
            -e PREFECT_API_KEY=${{ secrets.PREFECT_API_KEY }} \
            -v /var/run/docker.sock:/var/run/docker.sock \
            -v $PWD:/workspace -w /workspace \
            prefecthq/prefect:3-latest \
            bash -c "pip install 'prefect-docker>=0.3.1' && prefect deploy --name my-deployment"

Option B: Python script with

flow.deploy()

or

flow.from_source(...).deploy(...)

- Good when you want to keep deployment logic in Python or use git-based code storage. Python deploy script:

# deploy.py
from prefect import flow

@flow
def my_flow(x: int = 1):
    return x + 1

if __name__ == "__main__":
    my_flow.deploy(
        name="my-deployment",
        work_pool_name="docker-pool",
        # image=... (if you build/push your own image)
        # push=True, build=True,
        cron="0 6 * * *",
        parameters={"x": 42},
    )

Run it in CI via a temporary container:

docker run --rm \
  -e PREFECT_API_URL=${PREFECT_API_URL} \
  -e PREFECT_API_KEY=${PREFECT_API_KEY} \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v $PWD:/workspace -w /workspace \
  prefecthq/prefect:3-latest \
  bash -c "pip install -r requirements.txt && python deploy.py"

Notes for Oracle VM - It’s just your Docker host. Mount

/var/run/docker.sock

into the worker if you use the Docker work pool. - Ensure the worker can reach your Prefect API URL (Cloud or self-hosted). - If running self-hosted Prefect Server in Compose, you can keep it in the same stack and point

PREFECT_API_URL

to the service name and port. When to use which pattern

- Prefer Docker work pool + worker in Compose for isolation and reproducibility - Use a temporary container in CI to run

prefect deploy

or

python deploy.py

- If you want to avoid rebuilding images on code changes, use

flow.from_source(...).deploy(...)

with a git repo Docs you’ll likely need - Deployments: Deployments (concepts) - Create deployments: Create deployments (how-to) - Deploy via Python: Deploy via Python - Work pools: Work pools - Workers: Workers - Run flows in Docker: Run flows in Docker - Self-hosted with Docker Compose: Self-hosted with Docker Compose A couple of clarifying questions to tailor this: - Are you using Prefect Cloud or running a self-hosted Prefect Server on the Oracle VM? - Do you plan to run flows in ephemeral Docker containers (Docker work pool) or inside the worker container (Process work pool)? - Which CI system are you using (GitHub Actions, GitLab CI, Jenkins, etc.)?