You’re not wrong to be surprised — the important nuance is that in Prefect 3.x, result persistence and caching are separate knobs: -

PREFECT_RESULTS_PERSIST_BY_DEFAULT=false

only disables storing task results; it does not disable cache key computation. - If you don’t specify a

cache_policy

on a task, Prefect uses the default policy (

DEFAULT = INPUTS + TASK_SOURCE + RUN_ID

), which computes a cache key by hashing inputs — hence the serialization errors when inputs contain locks, file handles, etc. You do not need to add

NO_CACHE

to every task. Use one of these global settings instead: Option A — turn off caching everywhere (strongest) - Env var:

PREFECT_TASKS_DISABLE_CACHING=true

- Prefect profile (prefect.toml):

[tasks]
  disable_caching = true

This forces

NO_CACHE

for all tasks, even if a task tries to opt into caching. Option B — disable caching by default, but allow opt-in on specific tasks - Env var:

PREFECT_TASKS_DEFAULT_NO_CACHE=true

- Prefect profile (prefect.toml):

[tasks]
  default_no_cache = true

Then, for the few tasks you want cached, explicitly set a

cache_key_fn

or

cache_policy

(and turn on result persistence for them if you want cross-run cache re-use). Example of opting a single task into caching while excluding non-serializable inputs:

from prefect import task
from prefect.cache_policies import INPUTS

@task(
    persist_result=True,  # required if you want cache hits across runs
    cache_policy=INPUTS - "db" - "lock",  # exclude problematic args
)
def compute(db, lock, x):
    return x * 2

Helpful docs: - Caching in Prefect 3 - Results (persistence) - Settings reference If you tell me whether you want caching completely off or just off-by-default, I can share a ready-to-copy

prefect.toml

snippet for your setup.