Ankit
02/03/2023, 6:51 AMtask_runner=DaskTaskRunner(cluster_class=distributed.LocalCluster, cluster_kwargs={"n_workers": 8})
When I run the same code in jupyter notebook (as normal functions), tasks are getting completed in <1 minute whereas it's taking over 10 minutes to run in prefect with above task_runner. Can someone help with what might be wrong?14:41:17.588 | INFO | distributed.core - Event loop was unresponsive in Worker for 4.40s. This is often caused by long-running GIL-holding functions or moving large chunks of data. This can cause timeouts and instability.
Christopher Boyd
02/03/2023, 3:37 PMAnkit
02/04/2023, 8:25 AM