Hey everyone, do you know any resources for best practises / guides regarding production deployment of agents? I'm looking at https://docs.prefect.io/orchestration/agents/overview.html#agent-types and am wondering whether a single vm using a docker agent would be fine. I am looking for something with very little overhead as our team is very small (3 people, so I feel kubernetes agent is not really an option). Thanks!
k
Kyle McChesney
08/20/2021, 1:17 PM
If you are on AWS, the ECSAgent had pretty minimal overhead. There is a bit to manage and configure but overall it feels like a nice balance.
b
Bastian Röhrig
08/20/2021, 1:23 PM
Thanks, that does look interesting. Unfortunately we're on Azure and can't switch :/
k
Kevin Kho
08/20/2021, 1:43 PM
Yeah a single VM should be fine. For Local and Docker agent though, the Flows will run on it so it depends on how much resources your flows need.
✅ 1
b
Bastian Röhrig
08/20/2021, 3:16 PM
To expand on that question: Our proposed architecture would be prefect cloud + a single azure vm as an agent. Most of our prefect tasks would be just calls to trigger azure ML / azure Functions to do the heavy lifting and then stitch together the results. Does that sound reasonable or are there any obvious issues?
k
Kevin Kho
08/20/2021, 3:28 PM
That sounds reasonable. That’s how ECS is. In that case you only need a lightweight agent.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.