Another question! I notice that some of the tasks on my flow are running multiple times, even though they have finished successfully (so they finish successfully 5 times). What could cause such a behavior?

This is almost always caused by Dask rescheduling work; we already have a “version locking” feature in Prefect Cloud that we hope to make more widely available in the very near future that prevents your tasks from running twice will still preserving a clean user experience

thanks, you mention it runs twice but I see it’s running 5 times. have you seen such a behavior too?

are you using the

LocalDaskExecutor

by any chance?

yes I am

yea that executor is known to rerun tasks, but I’ll admit I’ve never heard of 5 reruns! Yea it’s a total different scheduler

so do you recommend I stop being lazy and move to

DaskExecutor

?

Yea I think that is the simplest solution to your rerun issue

@Chris White following up here. I am using

DaskExecutor

now. The scheduler and the worker themselves are still local, but I specify them in different processes in the system