In Prefect 3.x, the concepts of "workers" and "runners" serve distinct roles in orchestrating tasks and flows:
1.
Workers:
- Workers are lightweight polling services that retrieve scheduled runs from a work pool and execute them.
- Each worker has a type corresponding to the execution environment, such as Docker, Kubernetes, or ECS.
- Workers are responsible for provisioning the infrastructure needed to execute flow runs.
- Workers can only poll work pools that match their type, ensuring that the flow runs are executed in the correct environment.
- More details on workers can be found
here.
2.
Runners:
- Task runners in Prefect are responsible for executing the individual tasks within a flow.
- They manage the execution of tasks, ensuring that dependencies are respected and retries are handled if necessary.
- Task runners operate within the context of a flow run, orchestrated by the worker.
In summary, workers handle the execution of entire flow runs by provisioning the necessary infrastructure, while task runners manage the execution of individual tasks within those flows.