Hey everyone! I was looking over the docstring in the ECS worker, and noticed you can specify the
launch_type
to be
EC2
. This is great, because we've got heavy flows that just chew up memory, and Fargate is limited to 120GB. AWS has ec2 instances in the r8* family going up to 1.5TB in size, and I'd like to see if I can tell the workers to spin up something like an r8g.8xlarge instead of fargate. I can see how to specify the CPU and memory, but is there a way to specify the instance name/family/type?
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.