Hi all 🙂 Wanted to show how I deployed a worker in a docker environment while using Prefect Cloud. I am somewhat new to docker, so maybe someone can improve it, but I want to describe the challenges I face. What I wanted: Start a docker container which, upon initialization, logs in to my cloud account and automatically starts a Prefect worker. The problem: Prefect already has a docker image, but simply writing a Dockerfile like

FROM prefecthq/prefect:2-python3.11 

ARG PREFECT_TK

RUN prefect cloud login --key "${PREFECT_TK}" --workspace "mortenginnerupandreasenzohocom/homelab"

CMD prefect worker start --pool my-process-pool

does not work. The

CMD

line overwrites all the lines in the original prefect image, rendering it all useless. Appending the

CMD

to the

RUN

line with and

&&

does not work, as

RUN

is executed when the image is build, not deployed. Solution: Use multi stage builds. My Dockerfile looks likes this

# Install he prefect image and login.
FROM prefecthq/prefect:2-python3.11 AS builder

ARG PREFECT_TK

RUN prefect cloud login --key "${PREFECT_TK}" --workspace "mortenginnerupandreasenzohocom/homelab"

# Cant start worker from first stage, as CMD will override all CMD's in the original prefect image.
# Therefore, a second stage is added, where a CMD can be freely added without consequences for previous
# CMD's.
FROM builder AS starter

CMD prefect worker start --pool my-process-pool

and my compose.yaml looks like this:

services:
  worker:
    build:
      context: .
      args:
        PREFECT_TK: ${PREFECT_TK}  
    env_file:
      - .env

The prefect token is saved in a .env file. The image can then be build with command:

docker compose build

(sudo might be needed depending on how you installed docker)

thank you for sharing @morten!

no problem 🙂 hopefully others can use it.

FYI - this will bake your API key into the image… To avoid this I pass `PREFECT_API_KEY`env var via docker compose.

thanks @Oliver Mannion do you mean something like this?

services:
  worker:
    build:
      context: .
      args:
        PREFECT_TK: ${PREFECT_TK}  
    environment:
      PREFECT_TK

like this:

services:
  worker:
    build:
      context: .
    environment:
      - PREFECT_API_KEY
      - PREFECT_API_URL

makes sense - much better!