Short answer: yes. In Prefect 3.x, task “caching” is just persisted results looked up by a cache key. If you persist results to external storage (e.g., S3/GCS/Azure, a PVC/NFS, etc.), cached outputs survive pod restarts and can be reused by any worker that has access to the same storage. How it works (at a glance) - A task computes a cache key (via a cache policy or

cache_key_fn

). - On run, Prefect checks result storage for that key. If found and not expired, it returns the value without executing the task code. - By default, results are stored on the local filesystem of the run environment (not ideal for ephemeral K8s pods). - To survive pod crashes, enable result persistence and point

result_storage

to external storage. What to configure - Enable result persistence (globally or per task/flow). - Choose a cache strategy and optional expiration. - Set remote result storage (S3/GCS/Azure, etc.) so results live outside the pod. Minimal example (S3)

from datetime import timedelta
from prefect import flow, task
from prefect.cache_policies import INPUTS

# 1) Create and save an S3Bucket block once (e.g. in a setup script or REPL):
# from prefect_aws.s3 import S3Bucket, AwsCredentials
# s3 = S3Bucket(
#     credentials=AwsCredentials(aws_access_key_id="**", aws_secret_access_key="**"),
#     bucket_name="my-prefect-results",
# )
# s3.save("my-s3-bucket", overwrite=True)

@task(
    # Cache by inputs + expire after 1 hour
    cache_policy=INPUTS,
    cache_expiration=timedelta(hours=1),
    # Persist the result so cache can be reused across pods
    persist_result=True,
    # Store the result outside the pod
    result_storage="s3-bucket/my-s3-bucket",
)
def expensive_op(x: int) -> int:
    # ... heavy work ...
    return x * 2

@flow
def my_flow():
    a = expensive_op(21)
    b = expensive_op(21)  # will use the cached result if within expiration
    return a, b

Global settings (optional) - Persist all results by default:

PREFECT_RESULTS_PERSIST_BY_DEFAULT=true

- Set a default storage block for all results:

PREFECT_DEFAULT_RESULT_STORAGE_BLOCK=s3-bucket/my-s3-bucket

Notes and tips - You can also use

cache_key_fn

and

cache_expiration

directly if you prefer that style;

INPUTS

is a convenient cache policy for “same inputs → same cache.” - For Kubernetes, any remote store works (S3/GCS/Azure) or use a shared volume (PVC/NFS) if you prefer. - If you expect many concurrent runs with the same cache key across pods and want to prevent multiple executions racing to fill the same cache, you can configure stricter isolation and a distributed lock manager (e.g., Redis). This is optional; by default, Prefect allows concurrent runs and the last committer wins. - Advanced: you can separate the cache key storage from the result storage via

cache_policy.configure(key_storage=...)

if you want different locations for “where the cache key lives” vs “where the result lives.” If you share which storage backend you prefer (S3/GCS/Azure/PVC), I can tailor a ready-to-run snippet for that setup.