I was running a prefect flow but facing a run out of memory issue, how can I clear up and free the space. I saw a 17G .prefect file in the root directory, can I remove it?
k
Kevin Kho
11/10/2021, 9:25 PM
You can delete those files if you don’t need them. They are checkpoints from the flow runs. You can also turn off checkpoints with
Copy code
@task(checkpoint=False)
def ...
t
Tao Bian
11/10/2021, 9:41 PM
Thank you for your reply!! Can I clean the checkpoints for a specific flow since we have several flows ran.
k
Kevin Kho
11/10/2021, 9:42 PM
These checkpoints are mainly for restarting a flow run from failure. Instead of re-running the previous tasks, it will pull the checkpoint. So you can delete most of them safely
Kevin Kho
11/10/2021, 9:42 PM
Like just leave the most recent ones.
t
Tao Bian
11/10/2021, 9:47 PM
Thanks!!! So for accessing the checkpoints - I cd into .prefect/flows/name_of_my_flow , I see multiple files in there - is this the correct place to delte the checkpoints? and how do I know which one is the latest ?
k
Kevin Kho
11/10/2021, 9:49 PM
yes exactly. can you sort it by time?
t
Tao Bian
11/10/2021, 9:55 PM
Okay, can i remove the entire folder or remove all the checkpoints inside.
k
Kevin Kho
11/10/2021, 10:27 PM
All might be aggressive but as long as you dont have future flows that rely on these, yeah it should be fine
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.