Hello I m trying to set up a prefect deployment that utilize Prefect Community #ask-community

Hello! I'm trying to set up a prefect deployment t...

Emon Datta

11/26/2024, 7:42 AM

Hello! I'm trying to set up a prefect deployment that utilizes serverless compute via a Google Cloud Run Jobs push work pool. I have an existing docker image that I've created via cloudbuild at

<http://gcr.io/project_id/github.com/org/repo:prefect_id|gcr.io/project_id/github.com/org/repo:prefect_id>

. How can I specify for the deployment to use that existing docker image?

Emon Datta

11/26/2024, 7:45 AM

Alternately, I'm open to building a new docker image via prefect, though I need a custom Dockerfile since there are some special commandline tools the repo needs. I tried this method as well, but got opaque failures while installing requirements

The command '/bin/sh -c pip install --extra-index-url ${AUTHED_ARTIFACT_REG_URL} -r python/requirements.txt'

(I have used almost the same exact Dockerfile to deploy my cloud run service without issues, and I was able to build an image via cloudbuild using that custom Dockerfile).

Emon Datta

11/26/2024, 7:56 AM

for context, i'm getting the requirements error when running the following:

Copy code

my_flow.deploy(
        name="my-custom-dockerfile-deployment",
        work_pool_name="my-cloud-run-pool",
        image=DockerImage(
            name="my_image",
            tag="deploy-guide",
            dockerfile="path/PrefectDockerfile",
            platform="linux/amd64",
            buildargs={"AUTHED_ARTIFACT_REG_URL": "...."},

        ),
        push=False
    )

Brock

11/26/2024, 1:55 PM

I haven’t attempted this myself just yet, but have you see guides here? https://docs.prefect.io/v3/deploy/infrastructure-examples/serverless

Argus Chiu

11/26/2024, 2:52 PM

did this recently with a prefect.yaml file deploying to a self hosted prefect server on GCP. prefect.yaml

Copy code

name: prefect-test
prefect-version: 3.0.0

# Skip build and push since we're doing it manually
build: []
push: []

pull:
- prefect.deployments.steps.set_working_directory:
    directory: /app

deployments:
- name: hello-deployment
  version: null
  tags: []
  description: null
  schedule: {}
  flow_name: hello_flow
  entrypoint: flows/hello_flow.py:hello_flow
  work_pool:
    name: my-cloud-run-pool
    work_queue_name: default
    job_variables:
      image: northamerica-northeast1-docker.pkg.dev/prefect-test-xyz/prefect-test/hello-flow:latest
      platform: linux/amd64

🙌 1

Emon Datta

11/27/2024, 12:04 AM

Thank you @Argus Chiu!

👍 1

39 Views

Open in Slack

Previous Next