<@ULVA73B9P> does the prefect 2.x `ConcurrentTaskR...
# ask-marvin
g
@Marvin does the prefect 2.x
ConcurrentTaskRunner
use fork or spawn for multiprocessing
m
The
ConcurrentTaskRunner
in Prefect 2.x uses a thread pool maintained by
anyio
for concurrency. It does not use multiprocessing with fork or spawn; instead, it allows tasks to switch when blocking on IO. If you are looking for parallel execution using multiprocessing, you might consider using the
DaskTaskRunner
or
RayTaskRunner
for distributed task execution. For more details, you can visit the Prefect 2.x task runners documentation.
upvote 1