Hi all, question about checkpointing and caching:
There is a simple flow with two tasks: A -> B.
Caching is setup with cache_for=timedelta(days=1) and result=LocalResult('./tmp')
I expect that when I re run flow from console result would be computed once and then reused, but I see that each time I run flow new files are added to result directory.
Is this expected behaviour? How to cache data between flow runs?
c
Chris White
12/01/2020, 8:29 PM
Hi Andrey - are you running this against a Prefect backend (Server or Cloud)? Some of the caching functionality relies on persisted state in the Prefect database, so if you’re only running this interactively your cache might not persist between sessions
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.