Got a general question about configuring a flow to execute on AWS ECS EC2 instance instead of ECS Fargate. Followed this guide to deploy everything using the latter but have discovered I need more resources than what’s allowed for Fargate task definitions. Is there an example/documentation for creating tasks that will execute on a custom EC2 instance? Thanks much!

Hi @Svyat! Does your process run on Dask?

@Kevin Kho, no, at the moment it does not. It’s large read/transform operation that does involve loading a pandas data frame in memory

Hey @Svyat, you would need to make an EC2 instance and write your own task definition for ECS https://docs.aws.amazon.com/AmazonECS/latest/developerguide/getting-started-ecs-ec2.html

Understood, that’s very helpful. Thank you!