Hello I have a perhaps basic question but I can t seem to fi

Question

Hello I have a perhaps basic question but I can t seem to find what I need in the docs so I m asking it here I want to run my flows and agents from AWS ECR ECS I can t find a good step by step guide on how to do that so if anyone has any good guide for that let me know I have started with a simple hello world and want to run it on ECS I have docker ized this flow and pushed it up to ECR and pointed to that registry in a custom ECS block and have pointed to that ECS block when deploying my flow with `prefect deployment build flows ecs test flow py main n dev q dev ib ecs task first ecs test sb s3 first flow a` I have also tried just using the generic ` i ecs task` when deploying my flows to no avail In either case I get Submission failed KeyError No class found for dispatch key ecs task in registry for type Block when trying to create a quick run of the flow Am I going about this wrong How can I reliably deploy my flows to ECS

redsquare · Answer

Have you seen this repo <https github com PrefectHQ prefect recipes tree main devops infrastructure as code aws>

Max Jackson · Answer

I have which has been super helpful for deploying my prefect2 agent I want to deploy my prefect2 flow to ecs do you know of something comparable

redsquare · Answer

maybe <https github com anna geller dataflow ops blob main github workflows ecs prefect agent flows run in agent container yml>

Mike Grabbe · Answer

< Max Jackson> I used anna s dataflowops project as the template for our prefect cloud deployment in production She uses cloudformation templates to create the ecs cluster agent task and permissioning

Mike Grabbe · Answer

the two tricky parts are IAM permissions in your environment and getting the ECSTask block right

Max Jackson · Answer

Yeah I m unclear on how to best use the ECSTask block my end goal is just to run my python code in ECS not to run arbitrary commands and I m not sure how to call an ECS task block and pass code to it from within the same Python file It also looks like anna s project runs everything both flows and agents from the same ECR repo Is that correct If so is that the best way to go about doing things

Mike Grabbe · Answer

Well you don t get much benefit from ECS if you only run the agent container there with all flows running in process on the agent container

Mike Grabbe · Answer

The ECSTask block allows you to spawn new ecs tasks each of which can run a deployed flow

Max Jackson · Answer

Thank you our previous architect left the company without showing us around prefect 1 too much and now we re responsible for the transition sweat smile

Max Jackson · Answer

I think ideally we d like to run each of our flows on independent ECS tasks since that s what our previous prefect 1 set up has been doing

Mike Grabbe · Answer

Yup that s what we do as well

Mike Grabbe · Answer

It s working well for us I wish ecs task cold start times weren t a thing but it s not that bad

redsquare · Answer

problem with ecs is it downloads the image every single time no caching

redsquare · Answer

eks a far better option if you can

Mike Grabbe · Answer

That s what I told our devops team But they pushed hard for us to use ecs over eks

redsquare · Answer

ecs massive variance which tin you get on too cpu perf network speed all over the place