https://prefect.io logo
Title
m

Mihai H

11/29/2022, 1:30 PM
t

Tim Galvin

11/29/2022, 1:37 PM
I have seen this error when
dask-worker
s that are spun up by the
DaskTaskExecutor
are either paused by dask when it detects low memory, or when a
dask-worker
is killed somehow -- this last one happened to me when a
sig bus
error was thrown by the compute node I was running on.
:gratitude-thank-you: 1
m

Mihai H

11/29/2022, 1:54 PM
First of all thanks for your message
In my case memory is not an issue
So it could be the second
t

Tim Galvin

11/29/2022, 2:56 PM
I thought so for my case as well -- but there was something about the number of workers and exactly how my code was scaling that made dask pause a worker. In any case, it is something to look out for.
👍 1