When trying to build my flow (docker storage) with...
# ask-community
l
When trying to build my flow (docker storage) with
prefect build
I am running into an error
mkdir: cannot create directory '/opt/prefect/': Permission denied
- I am creating a user in my Dockerfile thru
useradd
it appears there are extra commands prefect appends to the user defined Dockerfile - is the root user required?
a
@Leon Kozlowski can you share your Dockerfile?
l
I can’t share the entire dockerfile but the useradd bit looks like this
Copy code
RUN useradd <name_of_user>

WORKDIR /home/<name_of_user>

RUN chown <name_of_user> /home/<name_of_user>

USER <name_of_user>
a
in general, if you specify the user after you set all the copying files and preparing environment, this should work fine. Example:
Copy code
FROM python:3.9-slim-bullseye
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY src/ .
USER 1000:1000
CMD ["python", "your_script.py"]
but for prefect, you don’t need to do most of that if you use the official images https://hub.docker.com/r/prefecthq/prefect
l
im using the prefect0.14.19-python3.8 as a builder
But based on this it looks like more commands are added to my dockerfile when using docker storage
a
So you use Docker storage? what you shared is basically adding health checks that ensure that your flow is serializable. You should be able to avoid the error you mentioned if you put the USER command near the end, because this user would only be defined for the process executing the flow run, rather than the process building the environment for the flow run
l
The user commands are at the end of my docker file - the
RUN mkdir -p /opt/prefect/
happens afterwards
This happens when I am trying to deploy my flow with
prefect build
a
I’d say that you don’t need any of this, i.e. neither the mkdir /opt/prefect/, nor prefect build, because when you register your flow, the Docker storage gets built anyway. But if you need those commands, then move them before the USER command:
Copy code
RUN mkdir -p /opt/prefect && prefect build
USER 1000:1000
l
the commands arent even in my dockerfile
prefect build
calls
storage.build()
which then calls
Docker
build and then
create_dockerfile_object
- thats where these commands are appended to my dockerfile
a
It seems like you want to have more control over the build process? if that’s the case, perhaps you can build the image independently on your own and pass it to DockerRun/KubernetesRun/ECSRun, and on registration not building the storage? Here is an example you could use. This way, whatever USER or image you choose, the Prefect’s storage.build() process won’t complain because you’re building the image on your own. You could use it as follows:
Copy code
if __name__ == "__main__":
    docker_storage.add_flow(flow)
    flow.register(project_name="community", build=False)
l
I didnt know building and pushing the image independently was an option - this is very helpful
🙌 1
thanks @Anna Geller
t
Hey @Anna Geller I'd like to know what build=False does ? Do you mind expanding on that ?
Copy code
flow.register(project_name="community", build=False)
a
@Tilak Maddy it registers a flow without building the Docker storage
usually (in 90% of the cases) you want your storage to be built upon registration, thus it’s set to True by default. But when you e.g. use Docker storage with
stored_as_script=True
, then building the storage upon registration makes no sense since this would pickle your flow rather than retrieving it from a script.
👍 1