Paul Gadal
10/07/2025, 9:47 AM-rw-r--r--. 1 root root 2565 Sep 18 14:35 ffffabc887a94443978d1739462dd51c
-rw-r--r--. 1 root root 2565 Sep 25 21:55 ffffe1353cfa46b8b27e6a4a20a82af7
-rw-r--r--. 1 root root 9191 Sep 28 08:26 fffffa5defe84203b79362073e1cdd3a
(base) [root@dtftlp11 storage]# cat fffd9f55adbb40df97262154386eff33
{"metadata":{"storage_key":"/data/.prefect/storage/fffd9f55adbb40df97262154386eff33","expiration":null,"serializer":{"type":"pickle","picklelib":"cloudpickle","picklelib_version":null},"pr
efect_version":"3.4.14","storage_block_id":null},"result":"gAWVkQYAAAAAAACMFnRibGliLnBpY2tsaW5nX3N1cHBvcnSUjBJ1bnBpY2tsZV9leGNlcHRpb26U\nk5QojBRkYXRhZmV0Y2gudGFza3MuY29yZZSMElJlbW90ZUZpbGV
5weZRoHIwabWV0ZW9mcmFuY2VfcmFkYXJfZG93bmxvYWSUaB5L\nAGgfSwBoICloIUsAaCJLAGgjS0BoJEsAdWJoJUsydWJoMksyaDNoJymBlH2UKGgqaAopgZR9lCho\nDX2UaA99lChoE
WhPaBNoUHVoFWgXKYGUfZQoaBpoU2gcaFRoHksAaB9LAGggKWghSwBoIksAaCNL\nQGgkSwB1YmglSzJ1YmgySzB1YnVidWJ1YnVih5RSlE6JTnSUUpQu\n"}(base) [root@dtftlp11 storage]#Marvin
10/07/2025, 9:47 AMMarvin
10/07/2025, 9:52 AMfrom prefect import flow, task
@task(persist_result=False)
def do_work(...):
...
@flow(persist_result=False)
def my_flow(...):
...
- Globally (recommended to ensure nothing persists by default):
prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=false PREFECT_TASKS_DEFAULT_PERSIST_RESULT=false
- Make return values JSON-serializable when possible, or explicitly return nothing, to avoid “auto-persist” when Prefect can’t inline the value.
B) Persist, but avoid local files
- Use a JSON serializer when your results are JSON-serializable; that lets Prefect inline data instead of writing a file:
- Per task/flow:
@task(result_serializer="json")
def small_result():
return {"status": "ok"} # JSON-serializable
- Or globally:
prefect config set PREFECT_RESULTS_DEFAULT_SERIALIZER=json
Note: Non-JSON values will still need storage (or will fail if you force JSON on them), so only do this if your results are JSON-serializable.
- Send results to remote storage with its own lifecycle/retention (S3/GCS/etc.) and let the bucket policy clean up:
prefect config set PREFECT_RESULTS_DEFAULT_STORAGE_BLOCK="s3/my-results-block"
Create/configure the block in the UI first and apply a lifecycle policy on the bucket.
C) Keep local files but auto-clean them
- Prefect doesn’t currently garbage-collect these files automatically (the expiration field you saw isn’t used for auto-deletion). Use OS tools or a small housekeeping job:
- Simple cron example (delete older than 7 days):
find /data/.prefect/storage -type f -mtime +7 -delete
- Or point local storage at a tmp location that gets cleaned:
prefect config set PREFECT_RESULTS_LOCAL_STORAGE_PATH=/tmp/prefect/storage
D) About caching
- If you’ve enabled task caching, that can also write to storage. You can disable caching globally if you don’t use it:
prefect config set PREFECT_TASKS_DISABLE_CACHING=true
You can inspect your current settings with:
prefect config view --show-defaults
Docs for details:
- Results and result storage: Prefect 3 Results
- Settings reference: Settings reference
If you can share whether you need to read results back from the UI/API, I can recommend the safest combination (e.g., turn off persistence entirely vs. switch to JSON or remote storage).