I can’t understand why I have a flow where task ru...
# ask-community
g
I can’t understand why I have a flow where task run more than one time. For example in the image there is a Parameter Task. Any suggestion?
k
Hey @Giovanni Giacco, did you register a flow with
flow.run()
inside?
g
I registered it to Prefect Cloud with flow.register() and executing it from the GUI
k
Could you share your flow code?
g
It’s a very long flow. I’m trying to create a minimum example.
Anyway I have not registered a flow with
flow.run()
inside and I noticed that with LocalExecutor it doesn’t happen but with Dask Executor and KubernetesRun yes.
k
Ah I see. Are you on Server?
g
I don’t know if I understood the question. I’m using Prefect Cloud.
k
Ah sorry. Yeah that was my question if you were on Prefect Cloud or Prefect Server.
So this can happen if Prefect submits tasks to Dask and then the Dask worker dies (can happen from memory issues). Dask itself has a failure handling mechanism that will restart the worker and restart the tasks on the worker. So what happens is Dask will attempt to retry tasks that have succeeded before already. For example, Prefect submitted tasks A, B and C to Dask and then run in the order A -> B -> C. If A and B succeeded, but then the Dask worker dies during C, Dask will try to start A and B again when it spins up a new worker. To avoid this, Prefect has Version Locking which will prevent those tasks from running again. If this is enabled on your Flow, then it won’t run those tasks again even if attempted
g
I didn’t know about Version Locking. Thank!🙏 Just to understand, is this feature available only for Prefect Cloud?
k
Yes exactly that’s why I was asking if you were on Server
g
I can’t activate the version locking from the GUI. The message is: “your team doesn’t have access to version locking”. Is a premium feature?
k
It is standard tier and above
👍 1