https://prefect.io logo
Title
s

Stefan

03/02/2023, 9:59 AM
Hi! The folder in .prefect is steadily growing in size - is this the default space for the database or is it some temp files that are not deleted?
1
t

Tim-Oliver

03/02/2023, 10:00 AM
storage
is where prefect caches your task-run results by default.
s

Stefan

03/02/2023, 10:02 AM
Oh. For representation in the UI or API, or for what purpose? They are wildly varying in size.
A task is only executed if the result does not exist yet.
If you run the same flow twice with the same parameters and with persisting results on the second run will use the cached results.
By default prefect uses pickle to save the returned values of a task as binary blob. The size of the files you see in
storage
depends on the size of the objects returned by your tasks.
s

Stefan

03/02/2023, 11:42 AM
OK I see. It's the storage of retries, results etc. How long are they persisted? Should we clean the folder?
t

Tim Galvin

03/02/2023, 2:41 PM
Is the persistent cache the default behaviour? In other words, are these binary blobs containing task run results unless told otherwise?
t

Tim Galvin

03/03/2023, 1:36 AM
Perfect! Thanks @Christopher Boyd