Ivar Zapata
11/25/2025, 9:31 AMMarvin
11/25/2025, 9:31 AMMarvin
11/25/2025, 9:34 AMpersist_result=True (sometimes referred to as “persist”) on a task only applies to that specific task. It does not cascade to other tasks that it calls. Propagation happens from flows to tasks, not from tasks to other tasks.
A quick illustration:
from prefect import flow, task
@task(persist_result=True)
def outer():
return inner() # inner does NOT inherit persist_result from outer
@task # no explicit persistence
def inner():
return 42
@flow # no explicit persistence at the flow level
def run():
return outer()
- outer() will persist its result.
- inner() will not persist its result unless you:
- explicitly set @task(persist_result=True) on it, or
- enable persistence at the flow level (see below), or
- have a global default set via settings.
Where propagation does happen:
- If you set @flow(persist_result=True), tasks called within that flow will, by default, persist results unless they explicitly override it.
Also, persistence and caching are different:
- persist_result controls storing results to disk/backends.
- caching is controlled separately (e.g., by attaching a cache policy). Enabling persistence on one task does not automatically “cache” other tasks.
If you’re seeing “all tasks cached,” it may be due to:
- persistence set at the flow level,
- global defaults like PREFECT_RESULTS_PERSIST_BY_DEFAULT or PREFECT_TASKS_DEFAULT_PERSIST_RESULT,
- or a cache policy applied to those tasks.
Docs for more detail:
- Results (persistence)
- Caching
- Settings and environment variables
If you can share a minimal snippet (flow/task decorators and any env vars) or confirm which Prefect version you’re on, I can pinpoint why you’re observing that behavior.Ivar Zapata
11/25/2025, 10:50 AMMarvin
11/25/2025, 10:50 AMMarvin
11/25/2025, 10:56 AMpersist_result=True on it due to:
- Flow-level propagation: @flow(persist_result=True) enables persistence for all tasks in that flow unless they override it.
- Global settings: environment or profile settings like PREFECT_RESULTS_PERSIST_BY_DEFAULT or PREFECT_TASKS_DEFAULT_PERSIST_RESULT.
- A cache policy on the task (cache policies require persistable results).
So if your outer task has persist_result=True and your inner task is persisting too, it’s typically because one of the above is causing the inner task to persist as well. That produces two keys:
- Key A: persisted result for the inner task
- Key B: persisted result for the outer task (which returns the inner task’s value, so both files often contain the same payload, but under different keys)
How to confirm what’s happening:
- Check your settings in code:
from prefect.settings import get_current_settings
print(get_current_settings().results.persist_by_default)
print(get_current_settings().tasks.default_persist_result)
- Make persistence explicit on the inner task to test:
@task(persist_result=False)
def inner():
return 42
With the outer task persist_result=True and inner persist_result=False, you should see a single new key (outer only).
If you can share:
- Your exact flow/task decorators (especially whether the flow has persist_result=True),
- Any cache policy usage,
- And whether you have any Prefect env vars set (e.g., PREFECT_RESULTS_PERSIST_BY_DEFAULT, PREFECT_TASKS_DEFAULT_PERSIST_RESULT),
I can pinpoint why you’re seeing two entries.
References:
- Results (persistence): https://docs.prefect.io/latest/concepts/results/
- Caching: https://docs.prefect.io/latest/concepts/caching/
- Settings: https://docs.prefect.io/latest/concepts/settings/Ivar Zapata
11/25/2025, 1:25 PMMarvin
11/25/2025, 1:25 PMMarvin
11/25/2025, 1:28 PMpersist_result in Prefect 3.x is:
1) The task’s own @task(persist_result=...)
2) PREFECT_TASKS_DEFAULT_PERSIST_RESULT
3) If neither is set, Prefect falls back to the current run context:
- If you’re inside another task, it uses the active TaskRunContext’s persist_result (i.e., the outer task’s value)
- Otherwise it uses the FlowRunContext’s value
- Otherwise it uses the global default (results.persist_by_default, which you have False)
So in your case:
- Outer task has persist_result=True
- Inner task has persist_result=None
- No task default and persist_by_default=False
→ The inner task picks up True from the outer task’s active TaskRunContext, so both persist and you see two keys in ~/.prefect/storage.
If you want only the outer to persist, set the inner explicitly to `persist_result=False`:
@task(persist_result=True)
def outer():
return inner()
@task(persist_result=False)
def inner():
return 42
Docs:
- Results (persistence): https://docs.prefect.io/latest/concepts/results/
- Settings: https://docs.prefect.io/latest/concepts/settings/Ivar Zapata
11/25/2025, 2:49 PMMarvin
11/25/2025, 2:49 PMMarvin
11/25/2025, 2:51 PM@task(persist_result=False)
def inner():
return 42
- Or disable it flow-wide and opt-in only where needed:
@flow(persist_result=False)
def my_flow():
return outer()
Quick self-checks:
- Print effective values during the run:
from prefect.context import TaskRunContext
from prefect.settings import get_current_settings
@task(persist_result=True)
def outer():
print("outer persist:", TaskRunContext.get().persist_result)
return inner()
@task
def inner():
print("inner persist:", TaskRunContext.get().persist_result)
return 1
- Verify Prefect sees your env var as expected:
from prefect.settings import get_current_settings
s = get_current_settings()
print("results.persist_by_default:", s.results.persist_by_default)
print("tasks.default_persist_result:", s.tasks.default_persist_result)
If your goal is “only the outer task persists,” the most reliable approach is to set @task(persist_result=False) on the inner task(s).