To persist data between flow runs in Prefect, you might consider using some of the following approaches:
1.
Using the Prefect API to Retrieve Flow Run Information
You can interact with the Prefect client in your flow to retrieve the start time or other metadata of the most recent flow runs. This
Client usage guide describes how to fetch flow run data using the API.
2.
Storing Results in the Database
For certain data types like booleans and nulls, Prefect can store the results directly in the database. Learn more about this in the
Results concept documentation. While there isn't a direct example of storing datetime of last run, storing results is the mechanism you would likely use.
3.
External Persistence Services like Redis
An alternative for persisting data between runs can be to use external services like Redis. While there's no built-in support for Redis in Prefect 2.x at the moment, here's a
Github issue discussing the idea which you might find insightful.
4.
Flow-Level Caching
There is an open issue about adding flow-level caching that could potentially be used to cache and retrieve results between flow runs. Check this
GitHub issue for updates on this feature request.
Based on these findings, a common pattern is to use external services or storage mechanisms to persist data which can be queried or retrieved in subsequent flow runs.
For your specific case of tracking the datetime of the last flow run, you might consider leveraging the Prefect API to fetch the most recent flow run data at the start of each flow and then saving that information somewhere persistent, such as a database or a file, using the result handlers.
Please refer to the
Prefect Docs on Results for more details on how to configure result handling and storage, and keep an eye on the related GitHub issues for updates on new features that might make this process more straightforward in the future.