Hello, I've noticed tasks start up, run and succee...
# ask-communitya
Andrew Pruchinski
07/22/2022, 5:13 PMHello, I've noticed tasks start up, run and succeed. Then start up again a few minutes later. Is this expected behavior? Is there a way to ensure this doesn't happen?
✅ 1
k
Kevin Kho
07/22/2022, 5:14 PM
Are you using DaskExecutor?
a
Andrew Pruchinski
07/22/2022, 5:14 PMYes
Andrew Pruchinski
07/22/2022, 5:15 PMthank you for the prompt response @Kevin Kho
k
Kevin Kho
07/22/2022, 5:16 PM
This is common if you submit A->B->C to Dask and then A succeeds and B succeeds but the worker dies during C. It will try to restart the worker and re-submit all of the work. Prefect will not re-run the work though. It will try to, but then it will see it’s already in a success state
a
Andrew Pruchinski
07/22/2022, 5:21 PMOh sweet. Thank you for the clarification!
k
Kevin Kho
07/22/2022, 5:23 PM
Assuming you are on Cloud* It might re-run on Server
a
Andrew Pruchinski
07/22/2022, 5:24 PMWe are running on the cloud
4 Views