https://prefect.io logo
#prefect-community
Title
# prefect-community
m

Matthias

02/25/2022, 1:34 PM
I am trying to replicate running this flow on my own laptop with Docker Desktop. I have created a docker image (but did not push it to e.g. DockerHub) and registered the flow just like in the script to Prefect Cloud. Started a docker agent and did a quick run from the UI. So far so good everything works. But, when I modify the Dockerfile so that it can run as non-root, I can no longer do a successful run. I get the following error from the agent:
Copy code
docker.errors.NotFound: 404 Client Error for <http+docker://localhost/v1.41/containers/f30a3e9241fab0d272c3f36eae16867487fb187b964c8b3f22bc8fd05d2aa4d0/json>: Not Found ("No such container: f30a3e9241fab0d272c3f36eae16867487fb187b964c8b3f22bc8fd05d2aa4d0")
and the flow is stuck in submitted state. Does anyone knows how to fix it? Just to make sure, the only difference between the successful run and the run stuck in submitted state is the fact that I added a non-root user to the image. https://github.com/anna-geller/packaging-prefect-flows/blob/master/flows_no_build/docker_script_docker_run_local_image.py
1
FYI, this is the Dockerfile:
Copy code
FROM prefecthq/prefect:0.15.6-python3.8

ENV PREFECT_HOME=/opt/prefect

WORKDIR $PREFECT_HOME

COPY ./flow.py $PREFECT_HOME/flows/flow.py

RUN useradd -m prefect-user && \
    # make it so that the directories in $PREFECT_HOME (including $PREFECT_HOME)
    # are world readable (and executable)
    chown -R prefect-user:root $PREFECT_HOME

USER prefect-user
ENTRYPOINT /bin/sh -c bash
Note that it looks different from the one in the tutorial from Anna Geller, but it works if I remove everything starting from the
RUN
statement
Nevermind, I solved the issue! When I remove the ENTRYPOINT, it works just fine for some reason...
🙌 1
a

Anna Geller

02/25/2022, 1:45 PM
Nice work! It makes sense because Prefect sets the ENTRYPOINT itself to trigger the flow run. I never set any entrypoints when building images for Prefect flows
m

Matthias

02/25/2022, 1:50 PM
That's very strange that Prefect does not overwrite the image's default entrypoint then, no?
a

Anna Geller

02/25/2022, 2:05 PM
I think it's the other way around 🙂 you use Prefect's base image and you actually have overwritten the entrypoint set by Prefect, that's why it didn't work for you. If you don't set it, you keep the entrypoint set in the Prefect's base image
m

Matthias

02/25/2022, 6:58 PM
That's true in this example, but the use case I have in mind is that I want to start from a regular Python base image. The reason for this is that some companies only offer a curated list of base images that you are allowed to use. Hence you cannot use Prefect’s base image and have to start from scratch. In that case, you want to make sure the entrypoint is properly set. In the job spec of the k8s runconfig, this is indeed the case https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/kubernetes/job_spec.yaml#L15
So an easy solution would be to add the same to the docker agent here https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/docker/agent.py#L473
a

Anna Geller

02/25/2022, 7:29 PM
I think to use your own custom base image, it would be easier to replicate Prefect's Dockerfile using your custom base image instead of changing commands used by Docker/Kubernetes agent, right? Also, the entrypoint is doing a bit more, e.g. allows to install packages set on env variable:
Copy code
pip install $EXTRA_PIP_PACKAGES
Lastly, Prefect is using the base Python image in the slim version, which is pretty standard and commonly accepted for Python applications - what else would you want to use instead (e.g. Ubuntu base-image and install Python yourself)? Having said that, feel free to submit a PR or GitHub issue and we can discuss it further with other engineers, if you feel like this is a problem we should address.
m

Matthias

02/25/2022, 7:55 PM
I’m guessing that indeed replicating Prefect’s Dockerfile solves the issue! Thanks for the advice!
👍 1
3 Views