I’m confused by what I thought was a core feature in Prefect – manual retry of a failed task in an o...

Simon

01/09/2024, 3:40 PM

I’m confused by what I thought was a core feature in Prefect – manual retry of a failed task in an otherwise successful flow. In Prefect 1, when a flow fails with some completed tasks and some failed tasks, and a manual flow retry is triggered in the UI, only the failed tasks start. This is nice. In Prefect 2, the entire flow restarts including all the completed tasks. This is bad, as these all take a long time to rerun.

Jenny

01/09/2024, 4:04 PM

Hi Simon! Thanks for the question. We’re working on an update for your issue - you should be able to follow along here: https://github.com/PrefectHQ/prefect/issues/10749

Simon

01/09/2024, 5:53 PM

Thanks Jenny, much appreciated

Simon

01/09/2024, 5:59 PM

Specifically this statement from

cicdw

there contains the answer:

anytime a completed state has an empty data payload Prefect decides to rerun the task

As none of our tasks by design return anything, Prefect just YOLO reruns them even when completed.

Jenny

01/09/2024, 6:01 PM

Actually, rereading this your issue should be fixed by turning on result persistence

Jenny

01/09/2024, 6:02 PM

https://docs.prefect.io/latest/concepts/results/

🙏 1

9 Views

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.