ESmith
12/16/2025, 8:03 PMMarvin
12/16/2025, 8:03 PMMarvin
12/16/2025, 8:05 PM@task functions inside an @flow creates new task objects each time the flow is called, so Prefect treats them as distinct tasks across runs.
- Fix: Define tasks at module level, then call them from the flow.
- Bad:
@flow
def my_flow():
@task
def do_work():
...
do_work()
- Good:
@task
def do_work():
...
@flow
def my_flow():
do_work()
- Calling the same task twice (once directly and once via submit)
- Symptom: Two runs of each task within a single flow run.
- Check for patterns like:
result1 = my_task(...) # runs as a task
result2 = my_task.submit(...) # runs again as a separate task
- Retries
- Symptom: Each task appears multiple times if it retries.
- If you’ve set retries= or retry_condition=, you’ll see additional task runs on failure/retry. Open a task run in the UI to see the attempt number.
- The flow is being executed twice
- Symptom: Everything doubles in a single trigger.
- Common when you both:
- Call the flow at import time or under if __name__ == "__main__": my_flow() and
- Also trigger it via a deployment
- For deployments, remove direct calls and use the deployment to run the flow.
- Dynamic fan-out (loops/mapping)
- If the second run’s input causes more iterations (e.g., a list doubles in size), you’ll see more task runs.
- Worker restarts / orchestration retries
- In rare cases (e.g., Dask/Ray/Concurrent task runners), a crash/restart can reschedule tasks. The UI can show multiple runs (one canceled, one completed).
Could you share:
- A minimal snippet of your flow and task definitions (especially where the tasks are defined)
- How you trigger the flow (deployment vs local run)
- Prefect version and task runner (e.g., Concurrent, Dask, Ray)
- A screenshot or description from the UI of how the 12 tasks appear (are there attempt numbers, or truly duplicated tiles)?
With that info, I can pinpoint the exact cause and suggest a precise fix.