morten
07/24/2023, 9:01 AMFROM prefecthq/prefect:2-python3.11
ARG PREFECT_TK
RUN prefect cloud login --key "${PREFECT_TK}" --workspace "mortenginnerupandreasenzohocom/homelab"
CMD prefect worker start --pool my-process-pool
does not work. The CMD
line overwrites all the lines in the original prefect image, rendering it all useless. Appending the CMD
to the RUN
line with and &&
does not work, as RUN
is executed when the image is build, not deployed.
Solution: Use multi stage builds. My Dockerfile looks likes this
# Install he prefect image and login.
FROM prefecthq/prefect:2-python3.11 AS builder
ARG PREFECT_TK
RUN prefect cloud login --key "${PREFECT_TK}" --workspace "mortenginnerupandreasenzohocom/homelab"
# Cant start worker from first stage, as CMD will override all CMD's in the original prefect image.
# Therefore, a second stage is added, where a CMD can be freely added without consequences for previous
# CMD's.
FROM builder AS starter
CMD prefect worker start --pool my-process-pool
and my compose.yaml looks like this:
services:
worker:
build:
context: .
args:
PREFECT_TK: ${PREFECT_TK}
env_file:
- .env
The prefect token is saved in a .env file.
The image can then be build with command:
docker compose build
(sudo might be needed depending on how you installed docker)Chris Reuter
07/24/2023, 11:26 AMmorten
07/24/2023, 11:27 AMOliver Mannion
07/25/2023, 7:16 AMmorten
07/25/2023, 8:19 AMservices:
worker:
build:
context: .
args:
PREFECT_TK: ${PREFECT_TK}
environment:
PREFECT_TK
Oliver Mannion
07/25/2023, 8:33 AMservices:
worker:
build:
context: .
environment:
- PREFECT_API_KEY
- PREFECT_API_URL
export PREFECT_API_KEY='xxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
export PREFECT_API_URL='<https://api.prefect.cloud/api/accounts/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/workspaces/xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx>'
prefect cloud login
in the Dockerfilemorten
07/25/2023, 8:37 AMPREFECT_API_URL
to local host (if I remember correctly), but that environment variable with the link would automatically take if that