I've been trying to run prefect based on a custom ...
# ask-community
p
I've been trying to run prefect based on a custom image. I have a rather complicated image build on a separate dockerfile. I wanted to just add prefect on top of that and cobbled up the following Dockerfile
Copy code
FROM [original-image]

RUN pip install prefect smbprotocol

# I keep the flow definition in the "prefect" directory
COPY prefect prefect

# The image wouldn't work without that
ENV TZ=Etc/UTC
ENV MKL_THREADING_LAYER=GNU
ENV MKL_SERVICE_FORCE_INTEL=1

# Run our flow script when the container starts
CMD ["python", "prefect/flow.py"]
So far I've been testing it in a local docker pool and all was well. I now moved to ACI and am getting entrypoint-related errors:
Copy code
Error: Failed to start container zuge6yxkhm5gm, Error response: to create containerd task: failed to create shim task: failed to create container 4d9adc93761405465a82af86e363832c0e7a043a584588e48a8ef0bb55faf532: guest RPC failure: failed to create container: failed to run runc create/exec call for container 4d9adc93761405465a82af86e363832c0e7a043a584588e48a8ef0bb55faf532 with exit status 1: container_linux.go:380: starting container process caused: exec: "/opt/prefect/entrypoint.sh": permission denied: unknown
I found that the ACI pool has a default entrypoint
/opt/prefect/entrypoint.sh
, so I cleared the entrypoint field, but no dice, I'm still getting the same error. I also tried to copy the entrypoint from prefect repo into the image (this one: https://github.com/PrefectHQ/prefect/blob/main/scripts/entrypoint.sh) but that still didn't work. Any suggestions? What's the intended way to run prefect based on a custom docker image? I want to avoid altering the original image as it's pretty difficult to build already.
It's pretty annoying to debug, I have to wait ~30-40 minutes between starting the flow and getting the error message 😞
this is how I tried to copy the entrypoint.sh script:
Copy code
RUN mkdir -p /opt/prefect && \
    curl -o /opt/prefect/entrypoint.sh <https://raw.githubusercontent.com/PrefectHQ/prefect/main/scripts/entrypoint.sh> && \
    chmod a+x /opt/prefect/entrypoint.sh
n
hi @Paweł Biernat 1. I would probably not name your directory
prefect
to avoid any potential namespace conflicts with the
prefect
package. 2. Why are you overriding the entrypoint? you generally don't have to do that, unless you're trying to customize how the prefect engine starts itself
p
1. Good point, thanks for noticing that. 2. I managed to make it work by adding the entrypoint.sh from prefect repo to my custom image. But I still don't know what's the recommended way to build an image not based on the official prefect image. Are there any examples where people do that? My Dockerfile now looks like this
Copy code
FROM xxx

RUN pip install prefect smbprotocol

RUN mkdir -p /opt/prefect && \
    curl -o /opt/prefect/entrypoint.sh <https://raw.githubusercontent.com/PrefectHQ/prefect/main/scripts/entrypoint.sh> && \
    chmod a+x /opt/prefect/entrypoint.sh

COPY prefect_pipeline prefect_pipeline
ENV TZ=Etc/UTC
ENV MKL_THREADING_LAYER=GNU
ENV MKL_SERVICE_FORCE_INTEL=1


ENTRYPOINT ["/opt/prefect/entrypoint.sh"]

# Run our flow script when the container starts
CMD ["python", "prefect_pipeline/flow.py"]
That works in both ACI pool and local docker pool. But I've got no idea if this is The Right Wayâ„¢ to do it.
n
I wouldnt say there is "one right way" to do it, what you're doing makes sense to me, I'm just not sure why you need to override the ENTRYPOINT or CMD in your case this example works fine for me, using python:3.12-slim