Thread
#prefect-community
    yu zeng

    yu zeng

    2 months ago
    hello. can i spec a arg to restart dask worker every hours and scatter the work's in-memory data to cluster before restart the worker ? i found my dask worker will oom when a long execution time, i can't find the mem-leak reason and so i have to restart works by a period of time, but when i restart worker it will loss the in-memory data and it will re-compute them, this waste time a lot and will make many mistakes.
    Anna Geller

    Anna Geller

    2 months ago
    I suggest asking here https://dask.discourse.group/ they know more about it
    yu zeng

    yu zeng

    2 months ago
    @Anna Geller i also ask them, but i guess some prefect commiter also commit to dask so i ask in here.
    Anna Geller

    Anna Geller

    2 months ago
    I'm no Dask expert, so they may help better, keep us posted then