Has anyone used the adaptive scale mode of Dask and DaskTaskRunner successfully? It is working for me, in so far as the tasks are running and are being completely. However, I am also seeing a set of short lived workers constantly being started and terminated. They start, connect to the dask schedular, and then are closed.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
