Hey 👋 I’m trying to figure out how to properly deploy Prefect on AWS ECS/Fargate such that we always have an agent running, ready to pick up any scheduled flow. I was able to execute

prefect agent fargate start -t $PREFECT_TOKEN

locally which appeared to work (it successfully started and was waiting for flows), but; 1. It never received any flows triggered from Prefect Cloud 2. This seems to be tied to my local terminal session (i.e. if I close my terminal, the agent shuts down) 3. I can’t see where this is actually deployed — I was expecting a new cluster to be created in AWS ECS, but that’s not the case Am I misunderstanding how this is supposed to work? If so, how can we go about deploying an always-running agent that’s ready to deploy new Fargate tasks whenever a flow is scheduled? Or is this not how Prefect is supposed to work either?

Hi Charles, I'm sorry you're having issues with this. A few quick notes:

It never received any flows triggered from Prefect Cloud

I'd check the labels configured on the agent/flows. These need to match (agents filter on labels) or your agent won't pick up any flow runs. See https://docs.prefect.io/orchestration/agents/overview.html#flow-affinity-labels for more info.

This seems to be tied to my local terminal session (i.e. if I close my terminal, the agent shuts down)

Yes, running

prefect agent <...> start

will start an agent process locally. Since many users have different tools they like to use for managing long running processes (k8s, docker, supervisord, cloud tools, etc...) we're agnostic to how you manage to keep the agent process running. Note that agents are stateless - killing an agent doesn't terminate any of the jobs started by it - you only need to have an agent process running when you want new jobs to be started.

I can’t see where this is actually deployed — I was expecting a new cluster to be created in AWS ECS, but that’s not the case

It will use your default cluster if one isn't specified.

Ok interesting! Looking forward to the ECS docs — this is likely how we’ll deploy.

you only need to have an agent process running when you want new jobs to be started.

If I have a flow that has a schedule to run daily at 02:00 in Prefect Cloud; my understanding is that I would need to either have an agent always running in ECS to receive the flow run daily at 02:00 and execute it — or can Prefect Cloud also start an agent? If

prefect agent ecs start

is tied to a terminal session, I don’t understand how I could have an agent that’s always running on ECS with this command. Either I run this command locally to spin up and ECS agent that’s tied to my local terminal session, or I manually deploy a long-running task in ECS and run the command

prefect agent local

within the ECS task.

can Prefect Cloud also start an agent?

Agents run on your infrastructure, you'd need to have one running to accept and start the flow run

I don’t understand how I could have an agent that’s always running on ECS with this command.

How you start a long running agent is up to you, there's too many different ways people manage infrastructure for us to support them all with quick CLI commands. In your case, you might start a long running ECS task that runs

prefect agent ecs start

, and has appropriate permissions to kick off other ECS tasks.

Hi @Jim Crist-Harif I want to follow up on this. I'm a bit confused on the "best" way to deploy prefect agents to ECS, as well. Do you have any examples of how to use terraform to deploying agents? Any examples/documentation you have on how to in-practice deploy an ecs cluster (and agents) would be great. Thanks

Hi @Marc Lipoff - I don't have any specific examples of using terraform to deploy agents but creating a long running agent with terraform shouldn't be different from starting a machine with an image and invoking a remote-exec command. So something like this:

resource "aws_instance" "web" {
  # the rest of your machine setup

  provisioner "remote-exec" {
    command = "prefect agent ecs start"
  }
}