https://prefect.io logo
m

Max Minerz

09/12/2023, 8:27 AM
Built Docker image not used by my worker (/opt/prefect/my_flows not found by worker, but findable locally in container) Hi all ! Hope you're doing well ! I'm currently stuck deploying my flows on my GKE cluster. Here is my configuration : I'm using
prefect deploy
from my
my_flows
directory containing my
prefect.yaml
file
Copy code
name: my_flows
prefect-version: 2.11.3

build:
  - prefect.deployments.steps.run_shell_script:
      id: get-commit-hash
      script: git rev-parse --short HEAD

  - prefect_docker.deployments.steps.build_docker_image:
      id: build-image
      requires: prefect-docker>=0.3.0
      image_name: my_repo/my_flows
      tag: "{{ get-commit-hash.stdout }}"

push:
  - prefect_docker.deployments.steps.push_docker_image:
      requires: prefect-docker
      image_name: "{{ build-image.image_name }}"
      tag: "{{ build-image.tag }}"
      credentials: "{{ prefect.blocks.docker-registry.docker-registry }}"

pull:
- prefect.projects.steps.set_working_directory:
    directory: /opt/prefect/my_flows

deployments:
  - name: process-lyrics-deployment
    version:
    tags: ["process-lyrics"]
    description:
    schedule: {}
    flow_name:
    entrypoint: process_lyrics/main.py:process_lyrics
    parameters:

    work_pool:
      name: pool-1
      work_queue_name:
      job_variables:
        image: "{{ image_name }}"
And this dockerfile
Copy code
FROM prefecthq/prefect:2.11.3-python3.8

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY . /opt/prefect/my_flows/
When I check my container locally, I can find everything in the directory
/opt/prefect/my_flows/
(as wanted) But when I run my job on my cluster I get this error...
Copy code
FileNotFoundError: [Errno 2] No such file or directory: '/opt/prefect/my_flows'
I'm using one single worker, with the same prefect version as my CLI (
prefect:2.11.3-python3.8
), and configured with
pullPolicy: Always
Since I've seen this forum post : https://discourse.prefect.io/t/how-to-configure-the-pull-step-for-a-project-building-a-docker-image/2777/2 But it doesn't seem to work for me... Thanks in advance for the help 🙏
👀 1
The problem is that my worker doesn't use my image as specified... but how can i check what's happening ?
j

Jack P

09/12/2023, 11:53 AM
@Max Minerz , kind of guessing sorry if it's wrong, but in deployments section of YAML add `path`:
Copy code
deployments:
  - name: process-lyrics-deployment
    ...
    entrypoint: process_lyrics/...py
    path: /opt/prefect
...
and if that fails try
path: /opt/prefect/my_flows
let me know if either of those work 🤞
b

Bianca Hoch

09/12/2023, 4:34 PM
Hey Max, I'm in the same boat as Jack, but let me spitball some ideas to see if we can get you unblocked. First things first, are you using an ARM-based chip (M1 or M2 Mac?) If so, you're going to want to add
platform: linux/amd64
to your
build_docker_image
step to make sure your docker image uses an AMD architecture.
m

Max Minerz

09/12/2023, 6:54 PM
Hi, thanks for your responses ! 🤩 Unfortunately none of these options have worked for me... I think the problem is that my flow is deployed on my workpool but without my built image 🥲 Is there a way to have logs for this ? An "image not found" or something like that ? Thanks a lot for your help
b

Bianca Hoch

09/20/2023, 3:23 PM
Hmm..are you pushing your images to a container registry? There's a section in this guide which goes over creating an artifact registry to host your custom image.
Enabling DEBUG level logging for the worker may help you out here, at least in terms of troubleshooting. You can run these commands in the worker's environment to enable them, and start up the worker process after the fact::
Copy code
prefect config set PREFECT_LOGGING_LEVEL=DEBUG
prefect worker start --pool my-pool-name
4 Views