Hi! A few months ago I designed a Flow using ECS run configuration, S3 for results and Docker for storage providing a registry_url in ECR. For the prefect agent, y created a service in AWS ECS (EC2 infrastructure). It worked correctly, but I am reproducing the same configuration for a new project and now it does not work: • Ir requires the awsvpc configuration. I solved that using the arg 

run_task_kwargs

 in the ECSRun configuration definition. • The arguments 

cpu

 and 

memory

  from the ECSRun configuration definition seems is not working. Even if I use valid values (cpu=256, memory=512) it creates the TaskDefinition using 1vcpu and 2GB RAM (I think it is the default configuration). I tried 

run_task_kwargs

 with the keyword 

overrides

 but it does not work neither. Do you know what I am doing wrong? Thank you very much!!

Hi @jcozar, we had a recent thread on this recently that may help.

Oh thank you very much! It’s true, the configuration is overwritten successfully. Therefore muy problem should be another one. I will debug deeply, thanks!

I think that’s right but let me ask someone with more ECS experience in the team and get back to you tomorrow

Hi @Kevin Kho! I have been the last few days reading about this. I think that the task definition is created with the default cpu and mem (1vcpu and 2GB) if

run_task_kwargs

is not used. Then, the

mem

and

cpu

override ithe values to the container, and therefore there is a task running in ECS consuming 1vcpu and 2GB, and a single container (the prefect flow) using just the overwritten values (i.e. 0.5vcpu and 1GB). I think that it would be very useful if the cpu and mem values are used internally to create the task definition instead of modifying the container values, because it will be a task with just one container (the prefect flow). Am I wrong or missing something about it? Thank you very much for your time!

Hello! I'm looking at the code, and looks like here if cpu/memory are provided in ECSRun, the overrides should define the cpu/memory for tasks, and not containers. If we would set it for container, it would be

containerOverrides

which according to AWS API run_task would set cpu/mem for container. Not sure where is the issue, let me try to run a few tests, and I'll get back to you

Hey @jcozar, did you ever find a way to override the network mode other than creating an entire task definition? I'm having similar issues...

Hi @Ben Muller, I didn’t. In the end I am using Fargate and a private subnet with a NAT, so the tasks in the private network can communicate with prefect through the NAT. I’m afraid that if you are using EC2 you need to define the whole task definition by yourself until the network mode is exposed as a parameter in prefect.

Ah ok, that's unfortunate, a pretty significant limitation imo. Hopefully they add that feature soon. Thanks for the reply @jcozar

You @Ben Muller can also create a VPC with the autoassign public IP option configured for IPv6 and check if it works for your use case.

Hey @jcozar, Mariia is out this week so I’ll take a look at the ECS stuff. Will get back to you.

Ok thank you @Kevin Kho, I’m not in a hurry with this, it’s just for helping 🙂 So it’s fine for me, and thanks for the info!

Thanks for letting me know 🙂 If I fail we’ll wait for Mariia lol.

haha I meant it’s fine for me if it takes more time to review it. You answered me many questions so I’m very grateful that you are looking for this 🙂