Channels
#prefect-community
Title
a
Andreas Nigg
07/22/2022, 9:36 AMHey. I just upgraded to prefect 2.0b11 and I've an agent running in a rather restrictive kubernetes environment - meaning I can't run it as root.
This results in having this warning at the Agent startup:
Copy code
/usr/local/lib/python3.9/site-packages/prefect/context.py:461: UserWarning: Failed to create the Prefect home directory at /.prefect
with SettingsContext(profile=profile, settings=new_settings) as ctx: Copy code
File "/usr/local/lib/python3.9/pathlib.py", line 1323, in mkdir
self._accessor.mkdir(self, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/.prefect/storage'ā
1
Copy code
from prefect import flow, task
from prefect.deployments import Deployment
from prefect.filesystems import RemoteFileSystem
from prefect.logging import get_run_logger
from prefect.packaging import FilePackager
@task
def greet_world():
logger = get_run_logger()
<http://logger.info|logger.info>("Hello world!")
@flow
def give_greeting() -> str:
greet_world()
minio_packager = FilePackager(filesystem=RemoteFileSystem(
basepath="<gcs://rm-datateam-prefect-global-storage>",
settings={
"type":
"service_account",
"project_id":
....
}))
Deployment(flow=give_greeting,
name="minio_file_package_with_remote_s3fs",
tags=["os4"],
packager=minio_packager)
if __name__ == "__main__":
give_greeting()(By the way sorry for the bad variable names... I use an object called "minio_packager" for GCS facepalm )
a
Anna Geller
07/22/2022, 9:51 AM
Thanks for reporting that, I assume this is the place where Prefect wants to store local results now but which didn't get created automatically
let me open an issue and ask the team why this folder
@Marvin open "2.0b11 throws
FileNotFoundError: [Errno 2] No such file or directory: '/.prefect/storage'm
Marvin
07/22/2022, 9:51 AMa
Andreas Nigg
07/22/2022, 9:54 AMHey @Anna Geller thanks for the reply.
My description was not enough, sorry.
ā¢ I use prefect 2.0 cloud
ā¢ I us an agent on kubernetes
In general, if the agent has the permissions to write (eg. if the kubernetes agent is run as root), then there is not a problem. However, if I can't run the agent as root, then I oftentimes don't have these write permissions.
I added these details to the github issue.
š 1
Just in this minute, I was able to workaround this issue, by manually creating a Dockerfile for the Agent, which manually sets permissions for this folder. Then it works. However it seems a little problematic to manually create a folder, than set permissions - already in the Agent build step.
Copy code
FROM prefecthq/prefect:2.0b11-python3.9
COPY requirements.txt .
RUN python3 -m pip install --upgrade pip && python3 -m pip install -r requirements.txt
RUN mkdir /.prefect && chgrp -R 0 /.prefect && chmod -R g=u /.prefectš 1
a
Anna Geller
07/22/2022, 10:28 AM
I updated it too - will ask the team, thanks a lot
a
Andreas Nigg
07/23/2022, 7:25 AMPerfect, thanks! This is a very practical solution.
š 1
9 Views