Hey all, quick question
So the default task_runner in prefect 3 is the Threadpool runner, what happens when we submit an async function ? is it still run within a thread ? what's the best way to execute async tasks in prefect 3 ?
Ouail Bendidi
01/27/2025, 10:46 AM
looking at the code, it seems like async tasks are indeed wrapped then executed within a thread:
Copy code
if task.isasync:
# TODO: Explore possibly using a long-lived thread with an event loop
# for better performance
future = self._executor.submit(
context.run,
asyncio.run,
run_task_async(**submit_kwargs),
)
else:
future = self._executor.submit(
context.run,
run_task_sync,
**submit_kwargs,
)
I can run them on the flow directly using:
Copy code
asyncio.gather(*(my_task(xx) for xx in xxs))
but then I would lose the
PrefectFuture
interface and the dependency between tasks in the UI
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.