Can you restart a failed Prefect run from a particular subflow/task (e.g. in the middle of the DAG)?
The compelling use case is a Data Science DAG that takes more than 20-30 minutes to run and you failed in the last mile (which takes <5 mins).
d
Deceivious
07/23/2023, 7:26 AM
No you cant.
But if you use caches and storage. You can skip the computation/ io and get the saved data from the cache instead.
o
Ofir
07/23/2023, 8:40 PM
Thanks @Deceivious.
Is there an example deployment / workflow code for using the caches and storage to skip and fetch data from cache?
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.