Hi there, we are using an ECS agent to run our flows. is it possible to set the cpu and memory per flow. I tried setting the cpu and memory for the flow in ECSRun config but the task that gets spun up still uses the default that was specified in the yaml file i.e cpu 1024 and memory 2048. i want to change the cpu and memory per flow. my current ecs run config is as follows:

RUN_CONFIG = ECSRun(
    labels=["dev"],
    task_role_arn="arn:aws:iam::xxxxx:role/prefectTaskRole",
    execution_role_arn="arn:aws:iam::xxxxx:role/prefectECSAgentTaskExecutionRole",
    image="<http://xxxxxx.dkr.ecr.us-east-1.amazonaws.com/prefect-orchestration:latest|xxxxxx.dkr.ecr.us-east-1.amazonaws.com/prefect-orchestration:latest>",      # this is our custom image that also has our flows
    run_task_kwargs=dict(cluster="prefectEcsClusterDev", launchType="FARGATE", overrides=dict(
        containerOverrides=[
            dict(
                name="flow",
                cpu=4096,
                memory=8192,
            )])),
    cpu=4096,                    #tried specifying here but still no luck
    memory=8192
)

Inside the

ECSRun(…)

will already have the highest priority and take effect. Did you re-register?

yes i tried re-registering the flow

This looks pretty right to me already. Will try it in a bit.

0.15.13

I didn’t see anything in the changelogs that would affect this behavior

@Kevin Kho any luck with getting this to work? just fyi we followed this tutorial to setup our ecs agent : https://towardsdatascience.com/how-to-cut-your-aws-ecs-costs-with-fargate-spot-and-prefect-1a1ba5d2e2df we also now have our own custom image that we deployed with our flows and other packages in it.

nearly there

you can click on task definition….then go to the json tab

So I see the default values in the task definition. I am wondering if this is overridden during run time. I left a message for the team and then will update you

thank you very much for digging deeper into this 🙏

This looks broken and there is an open issue. You’d need to use a task definition yaml for now

i do see a pr merged on that one is there a chance that this is fixed in the newer release?

No that PR merged is a separate thing to allow for SPOT instances