Hi - Under what scenarios would a flow take longe...
# prefect-community
i
Hi - Under what scenarios would a flow take longer to run locally with the DaskExecutor than the LocalExecutor? Is to many small tasks one of them?
c
Hi @itay livni - this is usually caused by two things: - having lots of small, fast tasks - exchanging large data between tasks Specifically, the slow down is typically caused by serialization overhead in sending data to / from the dask workers. We have identified a few ways of speeding this up under the hood, so those improvements should be coming out in the next week or so!
👍 1
i
Thanks 👍
@Chris White Is
0.12.5
faster? On one flow I have seen 10 to 30% improvement in runtime performance. ~18 sec to ~13.
upvote 4
c
yup it sure is! we simplified some of the information we were passing around, resulting in faster serialization times and a little less code
👏🏼 1
glad you noticed 😄
i
Kudos 💯