<@ULVA73B9P>, in a ECS Push configuration, how can...
# ask-marvin
b
@Marvin, in a ECS Push configuration, how can I set the storage size for a job?
m
thought for 108.5 seconds
Short answer: put it on the task definition. In an ECS Push work pool, set
ephemeralStorage.sizeInGiB
inside the
task_definition
of the pool’s Base job template (or override it per deployment via
job_variables
). Where to set it - In the UI: Work Pools → your ECS Push pool → Edit → Base job template. - Per deployment: pass
job_variables
to your
.deploy(...)
. Example Base job template snippet (pool-level)
Copy code
{
  "task_definition": {
    "family": "prefect-task",
    "cpu": "1024",
    "memory": "2048",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
      "FARGATE"
    ],
    "ephemeralStorage": {
      "sizeInGiB": 100
    },
    "executionRoleArn": "arn:aws:iam::123456789012:role/ecsExecutionRole",
    "taskRoleArn": "arn:aws:iam::123456789012:role/ecsTaskRole",
    "containerDefinitions": [
      {
        "name": "prefect",
        "image": "{{ image }}",
        "command": [],
        "environment": []
      }
    ]
  },
  "run_task_kwargs": {
    "cluster": "my-cluster",
    "launchType": "FARGATE",
    "networkConfiguration": {
      "awsvpcConfiguration": {
        "subnets": [
          "subnet-abc"
        ],
        "securityGroups": [
          "sg-xyz"
        ],
        "assignPublicIp": "ENABLED"
      }
    }
  }
}
Per-deployment override (Prefect 3.x)
Copy code
from prefect import flow

@flow
def my_flow():
    ...

my_flow.from_source(".").deploy(
    name="with-ephemeral-storage",
    work_pool_name="my-ecs-push",
    job_variables={
        "task_definition": {
            "ephemeralStorage": {"sizeInGiB": 100}
        }
    },
)
Notes (because AWS loves footnotes): - Works only for Fargate tasks; valid range is 21–200 GiB. Use platform version 1.4.0+. - If you need more or persistent storage, mount EFS via `volumes`/`mountPoints` in the task definition instead. Docs for reference: - Prefect ECS guide: Prefect 3 ECS integration guide - AWS ECS ephemeral storage: ECS Fargate task storage and RegisterTaskDefinition API