Hello, I have a question, i'd like to make a deployment from python directly (because i have multiple deployments based on booleans in a database) instead of prefect.yaml. I have used both flow.deploy and Deployment.build_from_flow but since I am using a custom docker image, i can't find a way to make it work. When i deploy using prefect.yaml it works well and I get the right pull_steps in my deployment like:

[
  {
    "prefect.deployments.steps.set_working_directory": {
      "directory": "/opt/prefect/..."
    }
  }
]

However using the two previous mentioned methods, it does not work Could someone shed a light on this pllease ?

Hi @Alexis 👋 here is how you would use

flow.deploy

with a custom Docker image, is that what you are looking for?

Hello Emil, thanks for your reply! Yes i've using that, however it doesn't seem to have the working directory as I want it to. When i compare to one deployed with the CLI, the "pull step" i wrote above is missing and i can't figure where it should be happening

When building the custom image, Prefect automatically manages injecting the flow code, so you shouldn’t have to specify a path. Do you need to specify a path?

Yes, here is my Dockerfile (maybe soemthing is wronge here to) :

RUN pip install --upgrade pip
RUN pip install poetry


WORKDIR /code
COPY pyproject.toml /code/

RUN apt-get update \
    && apt-get install -y libpq-dev gcc \
    && apt-get update && apt-get install ffmpeg libsm6 libxext6  -y \
    && apt-get install libglib2.0-0 -y



RUN poetry config virtualenvs.create false \
    && poetry install --no-interaction \
    && rm -rf /root/.cache/pypoetry

RUN mkdir /opt/prefect/highres-cd-processing

COPY . /opt/prefect/highres-cd-processing

My understanding is that you shouldn’t need to specify:

RUN mkdir /opt/prefect/highres-cd-processing

COPY . /opt/prefect/highres-cd-processing

In other words, as Prefect builds the image, it should inject the code itself. Try removing those lines and maybe also the

WORKDIR /code

and see what happens.

i will right away thanks 🙂

Hey @Alexis, I recommend adding a line to your Dockerfile to set the working directory to

/opt/prefect/highres-cd-processing

. Prefect will attempt to load your flow code relative to the current working directory when using a custom Dockerfile with

flow.deploy

.

Hey @alex,thanks for replying! I've tried this as well and I'm still having the same error. The task is pending for a minute or 2, and then it errors stating that it cannot find a file (my main function where the flow is located). I feel that I am very close but can't manage

Hmmm, I suspect it’s trying to load via entrypoint that we’re generating, but that entrypoint may not work in your Docker container because the directory structure isn’t what it’s expecting. @Alexis can you share the entrypoint for the deployment that is created? It should be relative to your

highres-cd-processing

processing folder.

yes the entrypoint is

run_images.py:run_highres_analysis

And the run_images.py is at the root of the docker normally (like a main.py)

Interesting. And this works with the same Dockerfile when your

pull_steps

look like this?

[
  {
    "prefect.deployments.steps.set_working_directory": {
      "directory": "/opt/prefect/highres-cd-processing"
    }
  }
]

Yes!

I’m surprised it didn’t work when you changed the working dir to

/opt/prefect/highres-cd-processing

in your Docker file. What type of infrastructure are you running your flows on?

They're running on Docker on our own servers (ubuntu)

Are you using a Docker worker to run containers for your flows?

yes I am !

OK, I think you might want to try changing the image pull policy on your work pool to Always. It’s possible that your instance didn’t pull a new version of your image when you updated the working directory. You can also try building with a new tag to ensure your instances pull the new image.

i'll give this a try soon, thanks a lot for your reply, will keep you updated 🙂