m

    Mikkel Antonsen

    1 year ago
    I have a last question and then I think I'm good to go. I have a local worker that calls code that uses multiprocessing. Daemonized processes aren't allowed to spawn children, so my code obviously fails. Is it possible to run code as a subprocess that is not daemonized, some how?
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Mikkel Antonsen, I do not believe so. Really, the easiest way do parallel process is to use the LocalDaskExecutor. If you are using either LocalDaskExecutor or DaskExecutor, think of it that all the cores or compute are already utilized to there is no room for a second stage multiprocessing.
    m

    Mikkel Antonsen

    1 year ago
    The multiprocessing is done by a library I'm using so it's "Outside" of my control
    Kevin Kho

    Kevin Kho

    1 year ago
    I think you’re limited to using the LocalExecutor then because Dask will really stop any multiprocessing code from running and raise the error