Hi everyone! 👋 I'm considering using Redis integration to distribute result persistence across multiple pods, enabling all flows to access the stored results. However, I encountered issues when trying to set a default TTL for result caching. Based on the documentation here: https://docs.prefect.io/v3/develop/task-caching, I observed the following: • Keys were not cleared over time, and the TTL was not applied using the

cache_expiration

argument. • Overriding the key name with the

cache_key_fn

argument did not work. Both features functioned as expected when using the default

WritableFilesystem

object (writing to the local file system). Has anyone experienced this or found a solution?

Hey @Oz Shaked! The

cache_expiration

currently lets Prefect know when to consider a result "valid" but doesn't control the removal of the stored results. It does make sense that we could control key expiration in the Redis via the

cache_expiration

kwarg. Would you be willing to open an issue in GitHub for this?

Hi @alex, Thanks for the fast reply. I Will open an issue. Do you want me to open one for implementing the

cache_key_fn

usage also in the

prefect_interface

? BTW, I looked at the code base and actually didn't see where you remove the files in the local system as well.

Yes please, an issue for the

cache_key_fn

bug would be great! Prefect doesn't remove any stored results right now, it only ignores results past their expiration. The expiration/removal would only be for Redis right now, but I could see us extending it to other storage mediums that support object expiration (like S3).