Hello! Is there a way to run flows solely on EC2 agents without using ECS? I've seen some past messages about running flows on EC2 with ECS, but my team is looking to only use EC2 and spin up/down instances based on a custom image on our end.

Yes, definitely there is a way to run your agent on EC2. You would then use a local agent. But note that Prefect agents are designed to be used as lightweight processes polling for work and they should be on 24/7. This means that the actual execution doesn’t have to run on the agent’s infrastructure. This page explains what I mean.

Great, I'll also check out the AWS batch path! More along the first path you brought up, is there a way we can create an EC2 service that would poll for flow runs, similar to what's described here and here, and spin up\down EC2 instances with local agent processes as needed for tasks in the flow?

so that’s what I was trying to explain and it looks like I failed 😄 You absolutely can spin up a local agent on EC2, BUT the agent cannot provision or start EC2 instances for you on demand because in order to start a flow run, the infrastructure for it must exist before the flow run can start. The only way to accomplish what you’ve described would be to leverage ECS Fargate (also EKS on Fargate) or AWS Batch, which do exactly what you want: before some job is submitted for execution, AWS spins up the compute resources required for the task at hand.

Okay, thanks Anna! I'll likely circle back with some more questions after reviewing the AWS Batch and ECS routes some more. I appreciate the help!