Hi! I am running into an issue where the agent is ...
# ask-community
d
Hi! I am running into an issue where the agent is running out of diskspace. It looks like the problem is that the PREFECT_LOCAL_STORAGE_PATH is filled up with persistant Results when running the flows. Some of my tasks are returning DataFrame’s which a significant amount of data which i assume creates the issue. So therefore, i am asking here on advice on how to proceed. I have thought along the following ideas: • Are there any ways i can limit (by stored bytes, or in time) how persistent the Results should be? • Otherwise, i guess the other option is to rewrite the flow so not the full dataframe is return. (For example dump the data frame to a temporary file which can be read by the next task.) • I guess another option might be to create a clean-up task in the end of the flow that delelets the stored results. This seems to me be the ugliest solution. What would you recommend? Or is there anything that i am missing?
j
I have a similar problem and have thought about writing not only a @task to clean up the files, but also a flow to delete all of the folders prefect creates when running a flow. Let me know if you come up with a better solution
d
Feels good to know that someone is running into the same issues 😉 Yeah, i think i will for now implement a clean-up task.
In this thread https://discourse.prefect.io/t/is-there-a-good-way-to-design-flows-for-restart-after-failure/1289/7 they are discussing the possibility to disable that results are persistently stored. @Anna Geller mentions that they have started working on that option - but i haven’t seen any further details on that. @Anna Geller would appreciate an update on this task? (or are there any ways we can track the progress?) 😃
a
thanks for asking; this is an ongoing work in progress and we plan to keep you updated about the progress via release notes and #CKNSX5WG3 channel I'll share with the team that this is important and urgent for you
d
Thanks! @Anna Geller
🙌 1
j
As Anna mentioned, an improvement is in the works. In the meantime you can set local storage to a tmp directory.
🙌 1