Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Does anybody encountered an issue with DaskTaskRunner where a task is re-run multiple times after successful completion?

Ok I think I got it. My Dask workers were too small, and some would die from time to time. When a Dask worker dies, it is restarted and run its task history once again (including completed tasks). Prefect only emits a warning when the worker is restarted, I wonder if this the desired behavior. Maybe it would be better if the flow crashed instead.