I can’t understand why I have a flow where task ru...
# ask-communityg
Giovanni Giacco
11/15/2021, 6:28 PMI can’t understand why I have a flow where task run more than one time. For example in the image there is a Parameter Task. Any suggestion?
k
Kevin Kho
11/15/2021, 6:29 PM
Hey @Giovanni Giacco, did you register a flow with
flow.run()g
Giovanni Giacco
11/15/2021, 6:30 PMI registered it to Prefect Cloud with flow.register() and executing it from the GUI
k
Kevin Kho
11/15/2021, 6:30 PM
Could you share your flow code?
g
Giovanni Giacco
11/15/2021, 6:31 PMIt’s a very long flow. I’m trying to create a minimum example.
Giovanni Giacco
11/15/2021, 6:43 PMAnyway I have not registered a flow with
flow.run()k
Kevin Kho
11/15/2021, 6:46 PM
Ah I see. Are you on Server?
g
Giovanni Giacco
11/15/2021, 6:48 PMI don’t know if I understood the question. I’m using Prefect Cloud.
k
Kevin Kho
11/15/2021, 6:49 PM
Ah sorry. Yeah that was my question if you were on Prefect Cloud or Prefect Server.
Kevin Kho
11/15/2021, 6:53 PM
So this can happen if Prefect submits tasks to Dask and then the Dask worker dies (can happen from memory issues). Dask itself has a failure handling mechanism that will restart the worker and restart the tasks on the worker. So what happens is Dask will attempt to retry tasks that have succeeded before already.
For example, Prefect submitted tasks A, B and C to Dask and then run in the order A -> B -> C. If A and B succeeded, but then the Dask worker dies during C, Dask will try to start A and B again when it spins up a new worker.
To avoid this, Prefect has Version Locking which will prevent those tasks from running again. If this is enabled on your Flow, then it won’t run those tasks again even if attempted
g
Giovanni Giacco
11/15/2021, 7:04 PMI didn’t know about Version Locking. Thank!🙏 Just to understand, is this feature available only for Prefect Cloud?
k
Kevin Kho
11/15/2021, 7:05 PM
Yes exactly that’s why I was asking if you were on Server
g
Giovanni Giacco
11/15/2021, 7:23 PMI can’t activate the version locking from the GUI. The message is: “your team doesn’t have access to version locking”. Is a premium feature?
k
Kevin Kho
11/15/2021, 7:28 PM
It is standard tier and above
👍 1
3 Views