Connor Keogh
11/01/2024, 4:45 AM--install-policy
is set to always
the worker will install the latest version of prefect + dependencies when starting up, which circumvents the pinned version. When I instead change the --install-policy
to if-not-present
, it's able to install the prefect[gcp]
dependencies fine but I keep getting the error
Unable to start worker. Please ensure you have the necessary dependencies installed to run your desired worker type.
after the dependencies have installed correctly:
"Successfully installed google-api-core-2.22.0 google-api-python-client-2.151.0 google-auth-2.35.0 google-auth-httplib2-0.2.0 google-cloud-core-2.4.1 google-cloud-secret-manager-2.21.0 google-cloud-storage-2.18.2 google-crc32c-1.6.0 google-resumable-media-2.7.2 googleapis-common-protos-1.65.0 grpc-google-iam-v1-0.13.1 grpcio-1.67.1 grpcio-status-1.67.1 httplib2-0.22.0 prefect-gcp-0.6.1 proto-plus-1.25.0 protobuf-5.28.3 pyasn1-0.6.1 pyasn1-modules-0.4.1 pyparsing-3.2.0 rsa-4.9 tenacity-9.0.0 uritemplate-4.1.1"
Kinda hard to ascertain what the issue is here, anyone else seen this? I'm deploying the worker via Terraform, with the following args
list:
args = [
"prefect",
"worker",
"start",
"--install-policy", "if-not-present",
"--with-healthcheck",
"-p", <my-workpool-name>,
"-t", "cloud-run-v2"
]
Daniel
11/04/2024, 5:42 PMDaniel
11/04/2024, 5:52 PMConnor Keogh
11/05/2024, 10:31 PMDaniel
11/05/2024, 11:51 PM# Use Python slim as the base image
FROM python:3.10-slim
# Set the working directory in the container
WORKDIR /app
# Install required system dependencies
RUN apt-get update --allow-releaseinfo-change && \
apt-get install -y --no-install-recommends \
curl \
gcc \
libffi-dev \
libssl-dev \
&& rm -rf /var/lib/apt/lists/*
# Install Prefect v2.19.9 and dependencies
RUN pip install --no-cache-dir prefect==2.19.9
RUN pip install prefect-docker
RUN pip install prefect-gcp
I think my issue was that I wasn’t explicitly installing prefect-dockerConnor Keogh
11/05/2024, 11:54 PMDaniel
11/06/2024, 9:02 AM