https://prefect.io logo
f

Francisco

12/09/2020, 8:27 PM
Hello Prefect Community! If a task fail... exist any way to retry from some upstream task? Example, I have three tasks: A->B->C If C fail, exist any way to re-run (retry) from task B? Or task A? Thanks in advance 😉
m

Mariia Kerimova

12/09/2020, 8:38 PM
Hello Francisco! You can restart from the failed task. Take a look at restart docs, also take advantage of using max_retries and retry_delay on your tasks.
f

Francisco

12/10/2020, 12:34 AM
Thanks, but the question is if can the flow retry from the upstream task, not the task that failed?
l

Lukas N.

12/10/2020, 9:59 AM
Have a look at checkpointing and
Result
. You can persist the output of task
B
(LocalStorage, S3, etc.) If task C fails, you can restart the flow and it will load the persisted output of task B, essentially skipping
A->B
computation. https://docs.prefect.io/core/concepts/results.html#pipeline-persisted-results
m

Matt Drago

12/14/2020, 5:32 AM
A hack that I did was to set the tasks that I wanted to re-run to a Failed state. When using the Restart Flow all of the failed tasks will be re-run. But be sure to set the state of all the tasks to re-run. So for A->B->C where C failed, but you want to re-run A and B as well, you need to set A and B to Failed. Note that I only did this as I am still in my experimentation phase of working with Prefect and I have not yet set up Results. I plan to setup Results properly to be able to restart flow correctly.