Rio McMahon
08/18/2022, 2:27 PMprefect agent start -q queue1 -q queue2 …
within the script below)
I am bootstrapping agents within a docker container with this python script:
import prefect
import asyncio
import subprocess
import time
import sys
async def bootstrap_agents():
# get list of work queues
pc = prefect.get_client()
work_queues = await pc.read_work_queues()
# create agents by ID
for wq in work_queues:
proc = subprocess.Popen(['prefect', 'agent', 'start', str(wq.id)])
print(f'Starting agent associated with {wq.name}. PID number is {proc.pid}', file=sys.stdout)
if __name__ == '__main__':
asyncio.run(bootstrap_agents())
while True:
time.sleep(1)
If there is a better deployment pattern I’d love to get insight on that.Ryan Lattanzi
08/18/2022, 2:37 PMProcess
right?Rio McMahon
08/18/2022, 2:42 PM-v /var/run/docker.sock:/var/run/docker.sock
so that although the agent is running within a docker container it can spawn adjacent containers (not docker in docker) with access to more compute than what is available within the container.
That said this is unknown territory for me so definitely looking for best practices.Ryan Lattanzi
08/18/2022, 2:54 PMJeff Hale
08/18/2022, 4:03 PMRio McMahon
08/18/2022, 4:27 PMJeff Hale
08/18/2022, 5:00 PM