We were considering upgrading from prefect 1.1 to 2.5. Given we rely on AWS ECS + Fargate, I'm concerned it might not be fully supported yet. It looks like 2.4 introduced ECSTask w/ a caveat that the api is fluid. Any additional color would helpful. We were only looking to update to stay relevant w/ community support, our 1.1 install has been stable.

The ECSTask is still experimental in 2.5 right now. If you're happy with your 1.1 configuration, you might want to stay with that until the ECSTask is out of experimental unless you want to start testing out migration.

Thanks, I think waiting makes sense. It is safe to assume that work is ongoing to get ECSTasks & Agents supported as first class components? Just trying to assess if I'm missing some directional shifts

I’ll just add that AWS ECS + Fargate appears to be working well for folks. In v2 it handles rate limiting better and doesn’t re-register the task unless it changes, unlike 1.0. It might be nice to start to migrate and pull the trigger when the experimental label goes away - or just pin your prefect-aws version to the current working version.

oh ok, if it's operational then maybe we can explore. Is there any documentation you suggest I read? I'm trying to figure out what is the equivalent of the

ECSAgent

Checkout this blog post by Anna, the ecs task docs. The agent is no longer a specific agent now with 2.0 and it is more generic. There are some helpful recipes for deploying the agent to ECS

Thanks, this was helpful. A few questions: 1. With the old ecs agent, you could provide

run_task_kwargs_path

which we used to specify a list of subnets. We would run 2 agents in 1 subnet. When an agent submitted a task, each submitted w/ a different set of subnets (private vs public) all in the same vpc. In the current model, I see how to set subnets for the agent, but not for where the agent launches a task. Looking at the source for EcsTask, I think it's only configurable via vpcId, but these subnet sets share the same vpc 2. If the agent requires the AwsCredentials block to launch the task, and the environment requires a sessionToken does this mean we need to run a process to periodically refresh the session token stored on disk for the block. I believe in the past this wasn't necessary because the ecs agent was long lived, and authenticated on startup. Now it seems the agents pick up flow runs from deployments which carry along a block with aws info like creds, so I'm not sure if the agent would remain aws authenticated 3. For different ecs task combinations of memory and cpu, we would need to have an ecs block for each, then bind that to the deployment. correct?

Hi Carlo. 1. Not sure. 2. That makes sense to me, but I’m not sure. 3. I believe so. @Anna Geller or @Zanie might be able to better answer.

1. There is a customizations field to override arbitrary items in the payload. I believe there is an example in the docs of providing subnets.

Thanks, I think I found docs for 1. https://medium.com/the-prefect-blog/prefect-2-5-makes-modular-and-serverless-dataflows-easier-than-ever-44d65b625627 For 2, I'll let you know if it's an issue once I get my agents going. For 3, yeah, I think it's multiple deployments. Honestly 2 & 3 might sort themselves once I clear my first hurdle.