https://prefect.io logo
#prefect-server
Title
# prefect-server
m

Mikkel Antonsen

09/14/2021, 9:09 AM
I have a last question and then I think I'm good to go. I have a local worker that calls code that uses multiprocessing. Daemonized processes aren't allowed to spawn children, so my code obviously fails. Is it possible to run code as a subprocess that is not daemonized, some how?
k

Kevin Kho

09/14/2021, 2:01 PM
Hey @Mikkel Antonsen, I do not believe so. Really, the easiest way do parallel process is to use the LocalDaskExecutor. If you are using either LocalDaskExecutor or DaskExecutor, think of it that all the cores or compute are already utilized to there is no room for a second stage multiprocessing.
m

Mikkel Antonsen

09/15/2021, 11:05 AM
The multiprocessing is done by a library I'm using so it's "Outside" of my control
k

Kevin Kho

09/15/2021, 1:54 PM
I think you’re limited to using the LocalExecutor then because Dask will really stop any multiprocessing code from running and raise the error
2 Views