Hi all, I'm currently running a simple flow with a Dask Executor that's spinning up a FargateCluster, but I'm getting a strange error that I can't find anywhere else online. When I run my flow with a DaskExecutor set up for a FargateCluster, I get a set of RunTimeErrors which say "IOLoop is closed". I've attached a .txt file of the code as well as the stack trace, if anyone has any advice it would be greatly appreciated! Thank you in advance

It looks like your flow ran successfully - the errors your seeing appear to be a bug in the shutdown process for dask-cloudprovider. I believe these are innocuous (prefect should shutdown your cluster explicitly, the errors are occurring because it's trying to shutdown again after the IOLoop has stopped). If you're a github user, would you mind opening an issue in the dask-cloudprovider repo for this? https://github.com/dask/dask-cloudprovider. You'll want to include your python and dask-cloudprovider versions. If not, I can open an issue, just let me know.

@Jim Crist-Harif thanks for the explanation. That's what I assumed was going on, but it was hard to verify since I'm not too experienced with the intricacies of Dask. I'll open an issue, thank you for pointing the in the right direction!

Hmmm, that's a more severe issue. Prefect manually calls

cluster.close()

when running flows, so if the cluster isn't shutting down properly this would still be a bug in dask-cloudprovider. I haven't seen this issue, it may specific to your environment (or your cluster configuration).

maybe it's an environment issue because the cluster config seems fairly similar to the defaults