https://prefect.io logo
Title
y

yu zeng

07/23/2022, 10:16 AM
hello. can i spec a arg to restart dask worker every hours and scatter the work's in-memory data to cluster before restart the worker ? i found my dask worker will oom when a long execution time, i can't find the mem-leak reason and so i have to restart works by a period of time, but when i restart worker it will loss the in-memory data and it will re-compute them, this waste time a lot and will make many mistakes.
1
a

Anna Geller

07/23/2022, 1:34 PM
I suggest asking here https://dask.discourse.group/ they know more about it
y

yu zeng

07/23/2022, 2:28 PM
@Anna Geller i also ask them, but i guess some prefect commiter also commit to dask so i ask in here.
a

Anna Geller

07/23/2022, 6:16 PM
I'm no Dask expert, so they may help better, keep us posted then