Hi. How the default

checkpointing

works in prefect 1.0.0? In documentation I see that it is enabled by default, but what is the default

Result

?

The default is a LocalResult that is Cloudpickled

@Kevin Kho Ok, thanks.

Input is not cached. Output is the result, which is the

.prefect

folder by default. Input cache was deprecated

so, Output caching and Result(checkpointing) is the same thing?

Not exactly, but when talking to people here they treat it synonymously. Caching is actually more about not running the task across Flow Runs while checkpointing is for restarting the same flow run from failure

But where Output Caching stores the cached data?

Still the Result because that is pulled for cached tasks. But the cache details (how long to cache for) are stored in the backend (Prefect Cloud)

I believe this documentation is outdated a little bit?

No no this is right. This is what I mean when I say checkpointing is different from caching

so, result arg is arg for both caching and checkpointing?

Think of it this way: 1. You can checkpoint a task. If the flow fails and restarts, you can load the checkpoint to resume (this has nothing to do with caching) 2. But if you start a new flow run, the checkpoints from the previous run don’t matter. Everything will run again. 3. So caching is the mechanism that lets you tell future runs not to run an expensive task again. 4. If a task is in a cached state, Prefect will pull the result if it exists in the future flow runs

ok, now it is more clear

That is 2 parts. S3 Result is stored in the S3. The cache with one hour expiry is stored on Prefect Cloud. So Prefect never deletes data. When the same task runs again in the future, it will check the cache and if the cache flag is not valid anymore, it will run the task again. So the Cached state is something we store in Cloud or Server

now it makes sense

Sorry my responses were too short cuz there’s just a lot to respond to this morning haha. So there was this feature there we held onto the inputs of a task execution so that when you restart from failure, they are held in memory. I think this was just changed to Prefect knowing where to get them from instead. I don’t know exactly, I just found it reading the source code but it’s before my time

Thank you for your time!!!

Of course!

Sorry, one more thing. You said that checkpoint only for retries, but what if checkpoint target will exist for new run?

If you mean this target. Target is a form of caching so that persists across flow runs. If you just do

Result(…, location=…)

, then this will not be respected

cool, thanks