Thank you very much to Laura for your help earlier! One more question regarding checkpoints and caching. Adding the

target

kwarg to the task caches the result as expected. I will try to contribute back to the documentation soon now that I have a better understanding of that vs a Result's

location

. Here is a modified version of the toy example I used before, this time I am trying to get it to forcibly ignore the cache and recompute. It is not considering the tasks' cache as still valid, despite a short

cache_for

and a validator of

never_use

. I know that

cache_for

and

cache_validator

in

Task

are deprecated. I tried specifying the validator on the

LocalResult

as you will see in the example.

from prefect import Flow, task, unmapped, Parameter
from prefect.engine.results import LocalResult
from prefect.engine.executors import DaskExecutor
from prefect.engine.cache_validators import all_parameters, never_use
import datetime

delta=datetime.timedelta(seconds=5)

lr = LocalResult(validators=[never_use])

@task(log_stdout=True, checkpoint=True, 
      target='{flow_name}-{task_name}-{parameters[x]}-{parameters[y]}.pkl',
      cache_validator=never_use, cache_for=delta)
def add(x, y):
    print(f'add ran with {x} {y}')
    try:
        return sum(x) + y
    except TypeError:
        return x + y

with Flow('iterated map', result=lr) as flow:
    y = unmapped(Parameter('y', default=8))
    x = Parameter('x', default=[1,2,3])
    mapped_result = add.map(x, y=y)
    out = add(mapped_result, y)

if __name__ == "__main__":
    flow.run(executor=DaskExecutor(),)

Hi! I think maybe you had a typo there, I think you are saying that “It is not considering the tasks’ cache as *IN*valid” despite your short cache_for key. Caching configured with `cache_for`/`cache_key` and caching with `target`s are separate right now (we do want to connect them though — the issue for that is https://github.com/PrefectHQ/prefect/issues/2619). Basically it is that we haven’t souped up targets all the way yet 🙂 . So any

cache_for

/

cache_key

and even

cache_validators

or

validators

will not affect target — just any old file existing at the

target

location will mean that the task is considered Cached, no other questions asked (currently). In addition, we check for the target if configured INSTEAD OF the cache used by cache_for/cache_key, so target always wins (again, only currently).

You are correct that I had a typo! But ok, good to know it’s not me misconfiguring something. I will take a look at the code and see if there is something I can contribute back to the core!

Awesome! And, we have a #prefect-contributors channel you can hang in too for contributing questions 🙂