Hello,
It might be a silly question but I can't re...
# ask-communityr
Romain
02/09/2022, 7:10 AMHello,
It might be a silly question but I can't really figure out the answer. One of the tasks of our flow is retried while it was previously already successful (see screenshot). And because this is a task calling a DELETE API endpoint, the retried task failed (because the object has already been removed). Although I could easily handle such thing, I would like to understand a bit more what's going on. I suspect something failed somewhere in the flow, and the flow is restarted from scratch, and all the tasks are restarted? I could not really find anything in the doc about that. What would cause a flow to retry a task that has been previously successful?
Some precision:
prefect core 0.15.12
prefect server 2022.01.12
k
Kevin Kho
02/09/2022, 7:13 AM
Are you on DaskExecutor?
Kevin Kho
02/09/2022, 7:17 AM
Dask has it’s own retry mechanism in the event a worker crashes or something like that. It will restart the worker and run the task graph again. On Prefect Cloud, we have a feature called Version Locking that will prevent the task from re-running. This is not in Server though
r
Romain
02/09/2022, 7:25 AMYes I m using a DaskExecutor. So it could be explained by a worker dying losing part of the task results, and the flow being required to be rerun?
k
Kevin Kho
02/09/2022, 7:27 AM
Yes. If Prefect submits A->B->C to Dask in one go, and then A succeeds and B succeeds, and then the worker dies, it will restart and believe it needs to do A and B because they are upstream of C.
👍 1
r
Romain
02/09/2022, 7:36 AMCrystal clear. Thanks
3 Views