i just had a task that ran twice. i only wanted it to run once and i got duplicate data as a result. i dont have retries enabled. is this something other people see sometimes? is there a way to avoid this?

Are you using Dask? and are you on Prefect Cloud? This is something like you had two jobs fire (Flows), or Dask recovering after dying can also re-submit tasks

i am on Prefect Cloud and im using a LocalDaskExecutor

There should be version locking to block the second execution. Does it actually run twice?

i believe so. i got duplicate data from it. how do i see if kubernetes spun up two jobs?

Check for two pods I think?

do i have to turn version locking on?

It should be on by default on Prefect Cloud. It is configurable per Flow and you can find it in the Flow settings

and the job completed so the pods arent up anymore

Oh, turning that on may help because it puts a lock on the task execution. Are you mapping? Yeah just do

query = """
mutation {
  enable_flow_version_lock(input: { flow_id: "your-flow-id-here" }) {
    success
  }
}
"""

from prefect.client import Client
client = Client()
client.graphql(query)

i am mapping (edit: im not mapping at the task level)

Let me read into that issue. I haven’t gone through it yet, but I think you are running into the issue.

i am running GKE autopilot so i also think i am. im also mapping tasks on other flows so even if im not seeing it here it is probably relevant

Any update to this thread? I have a similar problem (though I am using a DaskExecutor on ECS) and I have been unable to resolve it so far. The linked Git thread shed some light on the problem, but the only proposed solution seems to be dumping autoscaling, which would mean having to retool our CI which sets the cluster config when the flow is being registered. Not ideal.

Hey @Daniel Ross , no update to this thread and I think the Github thread is the best place to follow and that issue is being worked on. You can chime in there if you have additional details (and to follow it)

Thanks Kevin! I'll keep an eye on it.

i ended up moving away from autopilot and doing a standard cluster and i dont think im seeing the same issue anymore

Thanks for the update @Andrew Lawlor!