<@ULVA73B9P> I'm running a flow that submits 1000s...
# ask-marvin
d
@Marvin I'm running a flow that submits 1000s of tasks to a local dask scheduler on an ECS fargate container. I'm seeing errors early in the run like:
Copy code
distributed.client.FutureCancelledError: robust_download-0edaa48770c443d06d2ff0ad5e01883d cancelled for reason: scheduler-connection-lost.
Client lost the connection to the scheduler. Please check your connection and re-run your work.
I also notice that the Dask dashboard becomes unresponsive during that submission. Is there a way to avoid overloading the scheduler with task submission? Also, can I control the timeout/lost-connection settings for the scheduler?
@Marvin