https://prefect.io logo
Title
e

Emma Rizzi

09/29/2022, 1:25 PM
Hello! I use prefect 1 for flows that write data in a NFS, with Kubernetes and Docker runs, and docker storage for each flow. It seems that all flows run as root in their container as all files are owned by root when generated by the flow. I need to change the ownership of files generated, is it possible to change the user that flows execute as ? Thanks 🙂
1
c

Christopher Boyd

09/29/2022, 1:30 PM
HI Emma, this is an interesting question. You need to change the user that the executed jobs run as inside the container that is spawned? I’m guessing this is due to them writing to NFS and needing a user or be written as anonymous? This might be possible by changing the image and entry points / user , but I’d have to look at the job spec to see what might changeable
e

Emma Rizzi

09/29/2022, 1:39 PM
Hi Christopher, so far all the flows i have are simple python ETL flows, build with docker storage. I tried both building with a custom dockerfile and with base_image + python_dependencies. The specific reason is that our API serving the data do not run with root access and can't read the data. The options I have are changing the ownership of the files during generation, changing the permission to include the user of the API (seems doable only with a chmod everytime a new file is created, not best practice imo), or change the user of API to root (I would like to avoid that). If there's other options I'm interested too! I was first searching in the agent configuration but it seems i should actually change the Dockerfile. I just stumbled on the USER docker command. I tried adding "USER <my user>" with a user that exists on the VM the flow is executed on, but it fails as it does not exists inside the docker built. I think I need to dig more this option it looks close to what I need 🤔
c

Christopher Boyd

09/29/2022, 1:46 PM
you would need to add the user into the docker image specifically; RUN echo ‘nobody65534:65534:nobody😕:’ > /app/passwd && \ echo ‘nobody65534:’ > /app/group
would be an example of adding the nobody user, then setting the permissions
if it’s NFS though, and a specific user, you can chown to that userid I think even if it’s not actually mapped on the docker host?
e.g. if the user on the share is cboyd, and my uid is 31657, you could chown to 31657, and when you migrate it to nfs, it would be interpreted as cboyd
e

Emma Rizzi

09/29/2022, 1:49 PM
Interesting I will try that! I would not have tried to use the uid like that. I'll keep you updated with the tests 🙂
c

Christopher Boyd

09/29/2022, 1:49 PM
alternatively, I believe the dockerfile addition is:
RUN useradd --create-home --shell /bin/bash cboyd
e

Emma Rizzi

09/29/2022, 2:40 PM
@Christopher Boyd it worked! I paste here the commands I used for anyone seeing this thread :
extra_dockerfile_commands=[
        "RUN useradd -u 1001 myuser",
        "USER myuser"],
The systems interprets it as the correct user in the NFS it's perfect! Thanks a lot for your support 🙂
1
🙌 1