Hello Prefect Community! If a task fail... exist ...
# prefect-server
f
Hello Prefect Community! If a task fail... exist any way to retry from some upstream task? Example, I have three tasks: A->B->C If C fail, exist any way to re-run (retry) from task B? Or task A? Thanks in advance 😉
m
Hello Francisco! You can restart from the failed task. Take a look at restart docs, also take advantage of using max_retries and retry_delay on your tasks.
f
Thanks, but the question is if can the flow retry from the upstream task, not the task that failed?
l
Have a look at checkpointing and
Result
. You can persist the output of task
B
(LocalStorage, S3, etc.) If task C fails, you can restart the flow and it will load the persisted output of task B, essentially skipping
A->B
computation. https://docs.prefect.io/core/concepts/results.html#pipeline-persisted-results
m
A hack that I did was to set the tasks that I wanted to re-run to a Failed state. When using the Restart Flow all of the failed tasks will be re-run. But be sure to set the state of all the tasks to re-run. So for A->B->C where C failed, but you want to re-run A and B as well, you need to set A and B to Failed. Note that I only did this as I am still in my experimentation phase of working with Prefect and I have not yet set up Results. I plan to setup Results properly to be able to restart flow correctly.