Max Minerz
09/12/2023, 8:27 AMprefect deploy
from my my_flows
directory containing my prefect.yaml
file
name: my_flows
prefect-version: 2.11.3
build:
- prefect.deployments.steps.run_shell_script:
id: get-commit-hash
script: git rev-parse --short HEAD
- prefect_docker.deployments.steps.build_docker_image:
id: build-image
requires: prefect-docker>=0.3.0
image_name: my_repo/my_flows
tag: "{{ get-commit-hash.stdout }}"
push:
- prefect_docker.deployments.steps.push_docker_image:
requires: prefect-docker
image_name: "{{ build-image.image_name }}"
tag: "{{ build-image.tag }}"
credentials: "{{ prefect.blocks.docker-registry.docker-registry }}"
pull:
- prefect.projects.steps.set_working_directory:
directory: /opt/prefect/my_flows
deployments:
- name: process-lyrics-deployment
version:
tags: ["process-lyrics"]
description:
schedule: {}
flow_name:
entrypoint: process_lyrics/main.py:process_lyrics
parameters:
work_pool:
name: pool-1
work_queue_name:
job_variables:
image: "{{ image_name }}"
And this dockerfile
FROM prefecthq/prefect:2.11.3-python3.8
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY . /opt/prefect/my_flows/
When I check my container locally, I can find everything in the directory /opt/prefect/my_flows/
(as wanted)
But when I run my job on my cluster I get this error...
FileNotFoundError: [Errno 2] No such file or directory: '/opt/prefect/my_flows'
I'm using one single worker, with the same prefect version as my CLI ( prefect:2.11.3-python3.8
), and configured with pullPolicy: Always
Since I've seen this forum post : https://discourse.prefect.io/t/how-to-configure-the-pull-step-for-a-project-building-a-docker-image/2777/2
But it doesn't seem to work for me... Thanks in advance for the help 🙏Jack P
09/12/2023, 11:53 AMdeployments:
- name: process-lyrics-deployment
...
entrypoint: process_lyrics/...py
path: /opt/prefect
...
and if that fails try
path: /opt/prefect/my_flows
let me know if either of those work 🤞Bianca Hoch
09/12/2023, 4:34 PMplatform: linux/amd64
to your build_docker_image
step to make sure your docker image uses an AMD architecture.Max Minerz
09/12/2023, 6:54 PMBianca Hoch
09/20/2023, 3:23 PMprefect config set PREFECT_LOGGING_LEVEL=DEBUG
prefect worker start --pool my-pool-name