Charles Lariviere
12/07/2020, 3:52 PMprefect agent fargate start -t $PREFECT_TOKEN
locally which appeared to work (it successfully started and was waiting for flows), but;
1. It never received any flows triggered from Prefect Cloud
2. This seems to be tied to my local terminal session (i.e. if I close my terminal, the agent shuts down)
3. I can’t see where this is actually deployed — I was expecting a new cluster to be created in AWS ECS, but that’s not the case
Am I misunderstanding how this is supposed to work? If so, how can we go about deploying an always-running agent that’s ready to deploy new Fargate tasks whenever a flow is scheduled? Or is this not how Prefect is supposed to work either?Jim Crist-Harif
12/07/2020, 4:32 PMIt never received any flows triggered from Prefect CloudI'd check the labels configured on the agent/flows. These need to match (agents filter on labels) or your agent won't pick up any flow runs. See https://docs.prefect.io/orchestration/agents/overview.html#flow-affinity-labels for more info.
This seems to be tied to my local terminal session (i.e. if I close my terminal, the agent shuts down)Yes, running
prefect agent <...> start
will start an agent process locally. Since many users have different tools they like to use for managing long running processes (k8s, docker, supervisord, cloud tools, etc...) we're agnostic to how you manage to keep the agent process running. Note that agents are stateless - killing an agent doesn't terminate any of the jobs started by it - you only need to have an agent process running when you want new jobs to be started.
I can’t see where this is actually deployed — I was expecting a new cluster to be created in AWS ECS, but that’s not the caseIt will use your default cluster if one isn't specified.
prefect agent ecs start
). Besides api docs (https://docs.prefect.io/api/latest/cli/agent.html) there's nothing I can point you at (yet) for that. Docs coming soon!
The fargate agent will stick around for a few months though, so no rush to change, but it won't support the new run-config pattern (https://docs.prefect.io/orchestration/flow_config/run_configs.html) that we've moved to.Charles Lariviere
12/07/2020, 5:37 PMyou only need to have an agent process running when you want new jobs to be started.If I have a flow that has a schedule to run daily at 02:00 in Prefect Cloud; my understanding is that I would need to either have an agent always running in ECS to receive the flow run daily at 02:00 and execute it — or can Prefect Cloud also start an agent? If
prefect agent ecs start
is tied to a terminal session, I don’t understand how I could have an agent that’s always running on ECS with this command. Either I run this command locally to spin up and ECS agent that’s tied to my local terminal session, or I manually deploy a long-running task in ECS and run the command prefect agent local
within the ECS task.Jim Crist-Harif
12/07/2020, 5:59 PMcan Prefect Cloud also start an agent?Agents run on your infrastructure, you'd need to have one running to accept and start the flow run
I don’t understand how I could have an agent that’s always running on ECS with this command.How you start a long running agent is up to you, there's too many different ways people manage infrastructure for us to support them all with quick CLI commands. In your case, you might start a long running ECS task that runs
prefect agent ecs start
, and has appropriate permissions to kick off other ECS tasks.Marc Lipoff
12/11/2020, 3:52 PMnicholas
resource "aws_instance" "web" {
# the rest of your machine setup
provisioner "remote-exec" {
command = "prefect agent ecs start"
}
}