<@ULVA73B9P> Why I have an error, if I didn't use ...
# ask-marvin
r
@Marvin Why I have an error, if I didn't use persist_result=True in my flow or task : """ Error encountered when computing cache key - result will not be persisted. ... JSON error: Unable to serialize unknown type: <class 'pycdt.endpoints.notify.resource_decorator.ResourceNotifyDecorator'> Pickle error: cannot pickle '_thread.lock' object This often occurs when task inputs contain objects that cannot be cached like locks, file handles, or other system resources. To resolve this, you can: 1. Exclude these arguments by defining a custom
cache_key_fn
2. Disable caching by passing
cache_policy=NO_CACHE
"""
m
thought for 117.8 seconds
Short answer: In Prefect 3.x, caching is on by default for tasks. When caching is enabled, Prefect tries to compute a cache key by serializing your task inputs. That happens even if you didn’t set
persist_result=True
. If any input isn’t serializable (e.g., objects holding thread locks like your
ResourceNotifyDecorator
), Prefect logs the message you’re seeing and skips persistence for that run. More detail: - By default, tasks use a cache policy that includes inputs, source, and run id. Computing that cache key requires serializing your inputs. - When a cache policy is active (default), Prefect will auto-enable result persistence to support caching. - If serialization fails, you’ll see “Error encountered when computing cache key – result will not be persisted…”, but the task still runs normally; it just won’t reuse or persist a cached result. How to fix it (pick one): 1) Don’t need caching? Disable it for that task
Copy code
from prefect import task
from prefect.cache_policies import NO_CACHE

@task(cache_policy=NO_CACHE)
def my_task(...):
    ...
Or explicitly turn off result persistence (also disables caching):
Copy code
from prefect import task

@task(persist_result=False)
def my_task(...):
    ...
2) Want caching but exclude the problematic parameter - Using the cache policy to ignore a specific input name:
Copy code
from prefect import task
from prefect.cache_policies import INPUTS

@task(cache_policy=INPUTS - "resource")  # replace "resource" with your param name
def my_task(data, resource):
    ...
- Or define a custom cache key function that only hashes safe inputs:
Copy code
from prefect import task
from prefect.utilities.hashing import hash_objects

def my_cache_key_fn(ctx, params):
    safe = {k: v for k, v in params.items() if k != "resource"}  # exclude non-serializable
    return hash_objects(safe, raise_on_failure=True)

@task(cache_key_fn=my_cache_key_fn)
def my_task(data, resource):
    ...
3) Restructure so non-serializable objects aren’t passed as task parameters - Initialize those resources inside the task, or pass just primitive config needed to construct them. References: - Task decorator parameters (including
cache_policy
,
cache_key_fn
,
persist_result
): Task API reference - Cache policies and helpers: Cache policy API reference If you can share your task signature (which parameter holds
ResourceNotifyDecorator
), I can show the exact
INPUTS - "param_name"
or
cache_key_fn
you’d need.