Jackson Nguyen
01/10/2025, 11:50 PMMarvin
01/10/2025, 11:50 PMtask_runner
keyword in the flow decorator. While task runners are not strictly required for task execution, they enable more advanced execution patterns.
You can use different task runners for different tasks by creating nested flows. For example, you might use the ThreadPoolTaskRunner
for local concurrent execution and the DaskTaskRunner
for distributed execution on a Dask cluster.
Here’s a link for more details: Task Runners in Prefect 3.x.Nate
01/10/2025, 11:53 PM