<@ULVA73B9P>. I have a flow that has 6 tasks which...
# ask-marvin
e
@Marvin. I have a flow that has 6 tasks which completes fine. Then maybe the 2nd flow goes up to 12 tasks ( it somehow duplicates the initial 6 tasks ) why is that?
m
thought for 82.5 seconds
A few things can cause “duplicate” task runs in Prefect. Without seeing your flow code it’s hard to be sure, but these are the most common culprits to check: - Tasks defined inside your flow function - Symptom: First run shows 6 tasks, next run shows 12 (the original 6 plus 6 “new” ones with the same names). - Why: Defining
@task
functions inside an
@flow
creates new task objects each time the flow is called, so Prefect treats them as distinct tasks across runs. - Fix: Define tasks at module level, then call them from the flow. - Bad:
Copy code
@flow
    def my_flow():
        @task
        def do_work():
            ...
        do_work()
- Good:
Copy code
@task
    def do_work():
        ...

    @flow
    def my_flow():
        do_work()
- Calling the same task twice (once directly and once via submit) - Symptom: Two runs of each task within a single flow run. - Check for patterns like:
Copy code
result1 = my_task(...)         # runs as a task
    result2 = my_task.submit(...)  # runs again as a separate task
- Retries - Symptom: Each task appears multiple times if it retries. - If you’ve set
retries=
or
retry_condition=
, you’ll see additional task runs on failure/retry. Open a task run in the UI to see the attempt number. - The flow is being executed twice - Symptom: Everything doubles in a single trigger. - Common when you both: - Call the flow at import time or under
if __name__ == "__main__": my_flow()
and - Also trigger it via a deployment - For deployments, remove direct calls and use the deployment to run the flow. - Dynamic fan-out (loops/mapping) - If the second run’s input causes more iterations (e.g., a list doubles in size), you’ll see more task runs. - Worker restarts / orchestration retries - In rare cases (e.g., Dask/Ray/Concurrent task runners), a crash/restart can reschedule tasks. The UI can show multiple runs (one canceled, one completed). Could you share: - A minimal snippet of your flow and task definitions (especially where the tasks are defined) - How you trigger the flow (deployment vs local run) - Prefect version and task runner (e.g., Concurrent, Dask, Ray) - A screenshot or description from the UI of how the 12 tasks appear (are there attempt numbers, or truly duplicated tiles)? With that info, I can pinpoint the exact cause and suggest a precise fix.