I've been working through the tutorial and ran into a Dask KilledWorker error and was hoping someone might be able to point me to a solution or give me some pointers on how to debug this. I can start dask-scheduler and 2 dask-workers to successfully run the "Deployment: Dask" example. https://docs.prefect.io/core/tutorials/dask-cluster.html I can run the "Advanced Features" example through the DaskExecutor section where the DaskExecutor starts up the dask-cluster and process the URLs in 20% of the single threaded mode. https://docs.prefect.io/core/tutorials/advanced-mapping.html#outline If I add the scheduler address to the executor parameters

executor = DaskExecutor(address="<tcp://192.168.86.23:8786>")
flow.run(executor=executor)

I get the KilledWorker error. ERROR - prefect.Flow: x-files | Unexpected error occured in FlowRunner: KilledWorker('retrieve_url-a032a3d8-fa89-4131-ad4c-4f3c5fbd282c', <Worker 'tcp://192.168.86.23:58923', name: tcp://192.168.86.23:58923, memory: 0, processing: 1>) By commenting things out I'm down to the offending line being

html = requests.get(url)

in the retrieve_url function. Any thoughts?

Hi @Nate Atkins, KilledWorker is often a result of a worker running out of memory or hitting a segfault. (Dask is preventing the task from running again because it has previously killed a worker.) This link has more details: https://stackoverflow.com/questions/46691675/what-do-killedworker-exceptions-mean-in-dask Can you watch the Dask dashboard Workers tab for worker memory while your Flow runs? (Usually available on port 8787, e.g. http://localhost:8787 if you're running the Dask scheduler on your laptop.) Also see if there's any relevant output from the Dask workers.