Is caching done entirely based off of file existence? Has there been thought into other methods of result caching? I know there it's possible to add a

Serializer

but has there been thought into a

Verifier

or similar? (Entirely just curious at this point)

Hey Jackson! Currently there are two implementations of caching in Prefect, both of which technically support forms of validation: - the original form of Prefect caching: this requires you specify both

cache_for

and optionally a

cache_validator

on your task that you want cached; the cache validator can be an arbitrary function, but typically folks use one from the Prefect library. Options include input validation, parameter validation, and a few others. This form of validation works best when running against a Prefect backend in order that the

Cached

states are persisted - Task

target

- this keyword is for file existence based caching; however, it is more flexible than it appears. You can provide a custom callable function that accepts the task inputs, and all keys of Prefect context (a typical signature for these functions is

**kwargs

) and returns a string. In this callable you could implement any number of custom validation steps; the simplest is to template the filename based on the inputs to the task (which implicitly serves as a form of validation) but you could perform much more logic in the target callable than that if you want. Ultimately though, task targets do need to return a string representing the location of a file whose existence will be checked

I feel bad I didn't go check through the docs before I posted this question. 🙇 That said, this is really really cool and thanks for the explanation.

Yea you could definitely do that by using a custom target location that included prefect Dunder version (And no worries on the docs, I’m well aware there are scattered pieces of info that can be hard to discover - something we hope to fix soon!)