<@ULVA73B9P> i have successfully deployed a worker...
# ask-marvin
a
@Marvin i have successfully deployed a worker on gcp using the guide: gcloud run deploy prefect-worker --image=prefecthq/prefect:3-latest \ --set-env-vars PREFECT_API_URL=$PREFECT_API_URL,PREFECT_API_KEY=$PREFECT_API_KEY \ --service-account <YOUR-SERVICE-ACCOUNT-NAME> \ --no-cpu-throttling \ --min-instances 1 \ --startup-probe httpGet.port=8080,httpGet.path=/health,initialDelaySeconds=100,periodSeconds=20,timeoutSeconds=20 \ --args "prefect","worker","start","--install-policy","always","--with-healthcheck","-p","<WORK-POOL-NAME>","-t","cloud-run" it works well, but since my flow uses 3rd party libs it not prefect in the wroker, how do i ensure packages are preinstalled
m
thought for 464.1 seconds
Short answer: don’t put your flow’s dependencies in the worker. The worker just polls. Your flows run in separate Cloud Run Job containers, so the dependencies need to be in the flow-run image. You’ve got two good options: 1) Set a default image on your Cloud Run work pool (bake deps into it) - Build and push a custom image that includes your libs:
Copy code
# Dockerfile
FROM prefecthq/prefect:3-latest
RUN pip install --no-cache-dir -r requirements.txt
WORKDIR /app
COPY . /app
Copy code
gcloud builds submit --tag us-central1-docker.pkg.dev/PROJECT/REPO/prefect-flows:py310
- Make that image the default for the pool:
Copy code
prefect work-pool get-default-base-job-template --type cloud-run > cloud-run.json
# edit cloud-run.json -> set "variables.image" to "us-central1-docker.pkg.dev/PROJECT/REPO/prefect-flows:py310"
prefect work-pool update "<WORK-POOL-NAME>" --base-job-template cloud-run.json
Now every deployment that targets that pool uses your image (and your deps). 2) Specify the image per deployment (same idea, just scoped) - If you prefer code over JSON, set the image when deploying:
Copy code
from prefect import flow

@flow
def my_flow():
    ...

if __name__ == "__main__":
    my_flow.deploy(
        name="my-deployment",
        work_pool_name="<WORK-POOL-NAME>",
        image="us-central1-docker.pkg.dev/PROJECT/REPO/prefect-flows:py310",
    )
- Or let Prefect build/push for you:
Copy code
from prefect import flow
from prefect.docker import DockerImage

img = DockerImage(
    name="us-central1-docker.pkg.dev/PROJECT/REPO/prefect-flows",
    tag="v1",
    dockerfile="auto",  # uses your project's requirements/pyproject if present
)

@flow
def my_flow():
    ...

if __name__ == "__main__":
    my_flow.deploy(
        name="my-deployment",
        work_pool_name="<WORK-POOL-NAME>",
        image=img,
        build=True,
        push=True,
    )
Notes: - The worker flag
--install-policy always
won’t install your flow’s packages into the Cloud Run Job container. For containerized worker types (like cloud-run), bake dependencies into the image instead. - Make sure the Cloud Run Job’s service account can pull from your registry (Artifact Registry Reader) or you’ll get “Image not accessible” fun. Refs: - Flow.deploy API: Flow.deploy - Work pools overview: Work pools If you tell me where you host your image (Artifact Registry repo/region), I can give you an exact tag and command sequence.