Are there performance implications between • starting several agents each with their own single work queue • starting one agent with several work queues (e.g. moving all work queues to one agent like

prefect agent start -q queue1 -q queue2 …

within the script below) I am bootstrapping agents within a docker container with this python script:

import prefect
import asyncio
import subprocess
import time
import sys

async def bootstrap_agents():

    # get list of work queues
    pc = prefect.get_client()
    work_queues = await pc.read_work_queues()

    # create agents by ID 
    for wq in work_queues:
        proc = subprocess.Popen(['prefect', 'agent', 'start', str(wq.id)])
        print(f'Starting agent associated with {wq.name}. PID number is {proc.pid}', file=sys.stdout)

if __name__ == '__main__':
    asyncio.run(bootstrap_agents())
    while True:
        time.sleep(1)

If there is a better deployment pattern I’d love to get insight on that.

i don't work here - just asking you because we are exploring ways to set up agent(s) as well - is this script going to be run on something like an EC2 server - so multiple agents on one server? are your flow runs executed in docker containers or as subprocesses on the same machine? as for your question it seems like if there are many pieces of work in each work queue, then one agent could get overwhelmed? however, if you have a small number of flows that don't run super often, i can't imagine one agent couldn't handle it. i guess i asked about the flow run execution infrastructure because if you have a server with 8 cores, and you have 7 agents running, then there is only one core left to use for flow run executions if your infra is

Process

right?

Hi @Ryan Lattanzi - yeah that is the plan to run it on a VM. RE the 8 core comment: that could potentially be an issue but I am going to run this within docker with

-v /var/run/docker.sock:/var/run/docker.sock

so that although the agent is running within a docker container it can spawn adjacent containers (not docker in docker) with access to more compute than what is available within the container. That said this is unknown territory for me so definitely looking for best practices.

whew docker and cores confuses me haha - but if you can spin up more docker containers than cores available, then i don't see why you couldn't have several agents running in one container, with each agent spinning up new container(s) for flow execution. seems like it would scale better and you'd have more separation - i.e. if one agent goes down for some reason, the other 6 still can read from their respective queues - but if you have a single point of failure agent then everything would be on hold

@Rio McMahon, if you think you’ll need to be dynamically spinning up multiple containers for resource reasons, Kubernetes might be worth considering.

Hi @Jeff Hale - I don’t anticipate dynamically spinning up containers; my current plan is to tear down/rebuild the container running the agents anytime a work queue is added. That is the plan for now but I’ll keep K8's in mind. Any insight into the one agent/many queue vs many agents/one queue performance question raised above? Thanks

Thanks, Rio. I don’t think we have a best-practice recommendation on this at the moment, but let us know how things go.