Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hey everyone, hope everyone is having a great day, I am not able to figure out if persisting and caching can be used together or not or if cache can cache results in GCS or other repositories

Checkpointing and caching can be used together. If a task is cached during the next flow run, it will just pull the checkpointed result. You need to provide a location to your Result object to pull it from. 

There are two forms of caching. The cache for and cache validator is the first one. This is caching by time or inputs/outputs. The second one is targets. This is file based persistence. If the file exists, the result will just be loaded from it

Thanks Kevin, and sorry for late reply <https://docs.prefect.io/core/concepts/persistence.html#output-caching|if I only want to use the cache> currently there is not a way to specify strategy like GCP or AWS and targets in the docs

Both of these things exists for checkpointing so I am assuming cache is stored on prefect server or cloud

Tasks is a cached state will load the checkpoint so you would use the cache + result interface