Great question — you’ve got two pieces to solve: 1) make your self-hosted Prefect Server UI reachable to other machines, and 2) ensure all deployments/flows register to that same server so they show up at that URL. Below are the common setups and the key variables to set. Option A: expose the server directly on a host port - In your docker-compose for the server container: - Publish the port and bind the API to all interfaces - Example snippet:

services:
      prefect-server:
        image: prefecthq/prefect:3-latest
        command: ["prefect", "server", "start"]
        ports:
          - "4200:4200"            # host:container
        environment:
          - PREFECT_SERVER_API_HOST=0.0.0.0
          - PREFECT_SERVER_API_PORT=4200
          # If you want the UI to build links correctly:
          # - PREFECT_UI_URL=http://<host-or-ip>:4200/
          # - PREFECT_UI_API_URL=http://<host-or-ip>:4200/api/

- Open the firewall on RHEL9 if needed:

sudo firewall-cmd --permanent --add-port=4200/tcp
  sudo firewall-cmd --reload

- You can now reach the UI at: http//<host ip>4200 - All clients/workers should point to: http//<host ip>4200/api Option B: put the server behind a reverse proxy (recommended for HTTPS) - Terminate TLS at your proxy (nginx/traefik/etc.) and forward to the server’s 4200 - Configure prefect-server container with external URLs so the UI knows where to call the API:

environment:
    - PREFECT_SERVER_API_HOST=0.0.0.0
    - PREFECT_SERVER_API_PORT=4200
    - PREFECT_UI_URL=<https://prefect.myco.com/>
    - PREFECT_UI_API_URL=<https://prefect.myco.com/api/>

- In your proxy, route: - / → http://127.0.0.1:4200/ - /api/ → http://127.0.0.1:4200/api/ - Ensure standard X-Forwarded-* headers are passed - All clients/workers should point to: https://prefect.myco.com/api Pointing deployments and workers at this server (so all flows appear at that URL) - In Prefect 3.x, deployments register with whatever server your client is configured to use. - Set the API URL (note: must include /api): - Temporarily (per shell):

export PREFECT_API_URL=<https://prefect.myco.com/api>

- Or persist in your Prefect profile:

prefect config set PREFECT_API_URL=<https://prefect.myco.com/api>

- Deploy your flows: - From a project folder:

prefect deploy

- Or programmatically:

from prefect.flows import Flow

    Flow.from_source(
        source="<https://github.com/your-org/your-repo>",
        entrypoint="flows/my_flow.py:my_flow",
    ).deploy(name="my-deployment", work_pool_name="process-pool")

- Start your process worker on the RHEL9 host (this is good for local filesystem access):

export PREFECT_API_URL=<https://prefect.myco.com/api>
  prefect worker start --pool process-pool

- If you want basic auth on your self-hosted server: - On the server:

PREFECT_SERVER_API_AUTH_STRING=username:password

- On all clients/workers:

PREFECT_API_AUTH_STRING=username:password

- Do not set

PREFECT_API_KEY

(that’s for Prefect Cloud). Given your environment - You’re already running a process worker on RHEL9 and your flows need host filesystem paths — perfect. Keep running the worker on the host as you do now, just point it at the new server URL via

PREFECT_API_URL

. - 50–100 flows sharing the same Conda env is also fine; the process worker will use your existing Conda environment. Docs and examples - Docker Compose self-hosting guide: Self-hosted with Docker Compose - Server settings reference (UI/API URL, CORS, auth): Settings reference - Server overview: Prefect Server concepts - Workers: Workers explained Questions for you

- Do you want to expose the UI directly on a port, or put it behind your organization’s reverse proxy with HTTPS and a hostname (e.g., https://prefect.myco.com)? - Will you require basic authentication on the API? If you share your current docker-compose snippet (just the Prefect service section), I can suggest the exact changes.

Short answer: no — not if your compose service actually runs the worker. - If your compose.yml has a

prefect-worker

service whose command is

prefect worker start --pool maestro-process-pool

, that container is the worker. You do not need to start another worker process separately. - If your

prefect-worker

service does not run that command, you must add it as the container’s

command:

(or exec into the container and run it there). Important for your setup (host filesystem + shared Conda env on RHEL9) - Since your flows must access host paths and share the same Conda env, it’s usually better to keep the worker running on the RHEL9 host (outside of Docker) so it uses your existing Conda environment and has native filesystem access. In that case, yes, run:

export PREFECT_API_URL=https://<your-prefect-host-or-domain>/api
  prefect worker start --pool maestro-process-pool

- If you do run the worker in Docker, you must: - Mount all required host paths as volumes into the container - Ensure the container has the same dependencies as your Conda env (you’d typically bake them into the image) - Point it at your server:

services:
      prefect-worker:
        image: prefecthq/prefect:3-latest
        command: prefect worker start --pool maestro-process-pool
        environment:
          - PREFECT_API_URL=https://<your-prefect-host-or-domain>/api
        volumes:
          - /path/on/host:/path/in/container

Scaling workers - You can run multiple workers by scaling the compose service:

docker compose up -d --scale prefect-worker=3

or by running multiple host processes (each can target the same pool or specific work queues). Docs - Workers overview: Workers explained - Self-hosted with Docker Compose: Self-hosted with Docker Compose If you share your current

prefect-worker

service snippet, I can confirm whether it’s already starting the worker or suggest the minimal changes.

Absolutely — here’s a clean pattern using micromamba to bake your Conda environment into a Docker image that runs a Prefect worker. Project layout

.
├─ Dockerfile
├─ environment.yml
└─ docker-compose.yml

environment.yml (example)

name: flow
channels:
  - conda-forge
dependencies:
  - python=3.11
  - pip
  # Conda packages
  - pandas=2.2
  - pyarrow
  - fastparquet
  - sqlalchemy
  # You can add OS-backed libs via conda-forge when needed
  # - libxml2
  # - libpq
  - pip:
      - "prefect==3.*"
      # Optional extras for integrations
      # - "prefect-aws==0.*"
      # - "psycopg[binary]==3.*"
      # - your-private-package==*

Dockerfile

# Use micromamba to create a reproducible Conda environment
FROM mambaorg/micromamba:1.5.9

# Copy your conda env spec
COPY --chown=$MAMBA_USER:$MAMBA_USER environment.yml /tmp/environment.yml

# Create a named environment and clean caches
RUN micromamba create -y -n flow -f /tmp/environment.yml && \
    micromamba clean --all --yes

# Make the environment the default for CLI calls
ENV MAMBA_DEFAULT_ENV=flow
ENV PATH=/opt/conda/envs/flow/bin:${PATH}

# If you prefer to pin a specific Prefect version later, you can do:
# RUN micromamba run -n flow python -m pip install --no-cache-dir "prefect==3.0.0"

# Ensure the worker runs inside the conda environment by default
ENTRYPOINT ["micromamba", "run", "-n", "flow", "--no-capture-output"]
CMD ["prefect", "worker", "start", "--pool", "maestro-process-pool"]

docker-compose.yml (service snippet)

services:
  prefect-worker:
    build:
      context: .
      dockerfile: Dockerfile
    environment:
      - PREFECT_API_URL=<https://prefect.myco.com/api>   # <-- include /api
      # - PREFECT_API_AUTH_STRING=username:password     # if using self-hosted basic auth
    # Mount host paths your flows need to access
    volumes:
      - /data:/data
      - /configs:/configs:ro
    # Optional: if host file permissions matter, run as your host user
    # user: "${UID}:${GID}"
    # Scale out more workers if needed
    # deploy:
    #   replicas: 2

How this works - The image builds a Conda env named

flow

from your environment.yml and installs Prefect there (via pip in the env). - ENTRYPOINT ensures any command (including the worker) runs inside that Conda env. - Your worker will run flows in the same container and environment, so every flow shares the same baked Conda packages. - Mount host directories at the same absolute paths your flows expect so local filesystem access works transparently. Tips - Reproducibility: pin versions in environment.yml; consider conda-lock for locked builds if needed. - OS deps: prefer conda-forge packages for system libraries; it reduces the need to apt-get. - Upgrades: change versions in environment.yml and rebuild the image; then

docker compose up -d --build

. - Work pool type: ensure your work pool is “process” so the worker runs flows inside this container (sharing the baked env). - If you already run a worker on the RHEL9 host for native Conda + host FS, you can skip Docker entirely. But if you containerize, ensure all required host paths are mounted. If you share your actual environment.yml (or a list of your packages), I can validate it and tailor the Dockerfile for your stack.

Short answer: you don’t have to copy or mount your code if you deploy from Git. Otherwise, you can bake it into the image or mount it as a volume. Pick one of these patterns: 1) Git-based deployments (recommended) - Your deployments reference a Git repo; the worker clones the repo at runtime. No COPY or volume mount for code is needed. - Ensure your image contains git. - Example deployment:

from prefect.flows import Flow

  Flow.from_source(
      source="<https://github.com/your-org/your-repo>",
      entrypoint="src/my_flow.py:my_flow",
  ).deploy(
      name="my-deployment",
      work_pool_name="maestro-process-pool",
  )

- Worker container: no code volume needed; just point to the API.

services:
    prefect-worker:
      image: your-prefect-image:latest
      command: prefect worker start --pool maestro-process-pool
      environment:
        - PREFECT_API_URL=<https://prefect.myco.com/api>
      # mount only data/config volumes your flows need
      volumes:
        - /data:/data

- Tip: add git to your Conda env or base image so

git_clone

can run: - environment.yml: add

git

- or in Dockerfile:

RUN micromamba install -y -n flow -c conda-forge git

2) Bake code into the image (no runtime pull) - COPY your src into the image; rebuild the image when code changes. - Dockerfile (building on the micromamba pattern you’re using):

FROM mambaorg/micromamba:1.5.9

  COPY --chown=$MAMBA_USER:$MAMBA_USER environment.yml /tmp/environment.yml
  RUN micromamba create -y -n flow -f /tmp/environment.yml && micromamba clean --all --yes

  ENV MAMBA_DEFAULT_ENV=flow
  ENV PATH=/opt/conda/envs/flow/bin:${PATH}

  WORKDIR /opt/app
  COPY --chown=$MAMBA_USER:$MAMBA_USER src/ /opt/app/src/
  # If you keep a prefect.yaml or project files, copy them too:
  # COPY --chown=$MAMBA_USER:$MAMBA_USER prefect.yaml /opt/app/

  ENTRYPOINT ["micromamba", "run", "-n", "flow", "--no-capture-output"]
  CMD ["prefect", "worker", "start", "--pool", "maestro-process-pool"]

- Compose: no code volume needed in prod; still mount data volumes as needed. 3) Mount your src as a volume (handy for rapid local dev) - The container sees your local code; no image rebuild on each change. - Compose:

services:
    prefect-worker:
      build: .
      command: prefect worker start --pool maestro-process-pool
      environment:
        - PREFECT_API_URL=<https://prefect.myco.com/api>
      working_dir: /opt/app
      volumes:
        - ./src:/opt/app/src:rw
        - /data:/data

- Caveat: volume paths/permissions must match; not ideal for production. Which should you use? - If your workers are containerized and you want repeatability: use Git-based deployments or bake code into the image. Git is easiest to update and version; baked images are fully self-contained. - For your requirement to access host filesystem paths: regardless of how you ship code, ensure you mount those host directories into the worker container at the same paths your flows expect. - If your worker runs directly on the RHEL9 host (outside Docker), then your local code and Conda env are already available and you don’t need Docker COPY or volume mounts at all. Docs - Storing and pulling flow code: Store flow code for deployments If you share which route you prefer (Git pull vs baked image), I can tailor the exact Dockerfile/compose snippet for your setup.