Hello everyone, I got some questions regarding the Prefect Agent. Firstly, how should I choose the number of agents, is there a way to calculate or common way to decide this number? Secondly, is a single agent enough to handle (maximum) 1000 querys per hour, the server is running with a local environment and using a DaskExecutor(a local Dask cluster)
08/17/2020, 3:50 AM
Hi Ame - all agents do is submit flow runs for execution; they are not responsible for anything more than that. Individual tasks within a Flow are handled entirely by the Flows’ executor, which could vary from Flow to Flow. This means that a single agent can easily service many many Flows without issue.
The main reasons for creating multiple agents are:
- sending flows to different execution environments through the use of labels; e.g, some flows run in one kubernetes cluster, whereas others run on a local development laptop. Each one of these environments will need an Agent responsible for submitting into it.
- scaling out: if you anticipate flooding the execution environment an agent is running in, it might be beneficial to duplicate the agent. For example, a Docker agent on a single machine will submit flows to run as Docker containers. If you anticipate having so many flows running at one time that the Docker daemon / machine running those flows will become resource starved, then you might consider creating a duplicate agent running on another machine.
The second point is much more use-case specific. For example, I can run hundreds of Flows that do nothing more than sleep on a single computer simultaneously. But if those Flows were heavy from a resource usage perspective (CPU / memory), then I might need to scale out much sooner.
08/17/2020, 5:33 AM
Hey @Chris White,
Just a follow up query on second case. Is it possible to run multiple agents targeting same flow now?
The last time I tried this, I ran into issues where agents were trying to update state for same flow.
08/17/2020, 5:35 AM
Hi Sandeep, yes that is possible; both Prefect backends use a version lock to ensure only one agent gets each flow run
08/17/2020, 6:14 AM
08/17/2020, 11:43 AM
Hey @Chris White, thanks for the answer. So usually the agent capacity should not be the bottle neck on a local machine, and a single agent should be enough to serve a 1000 QPH, if the system resource is enough. Is this correct?