Bryan
08/19/2025, 9:28 PMMarvin
08/19/2025, 9:28 PMMarvin
08/19/2025, 9:30 PMephemeralStorage.sizeInGiB
inside the task_definition
of the pool’s Base job template (or override it per deployment via job_variables
).
Where to set it
- In the UI: Work Pools → your ECS Push pool → Edit → Base job template.
- Per deployment: pass job_variables
to your .deploy(...)
.
Example Base job template snippet (pool-level)
{
"task_definition": {
"family": "prefect-task",
"cpu": "1024",
"memory": "2048",
"networkMode": "awsvpc",
"requiresCompatibilities": [
"FARGATE"
],
"ephemeralStorage": {
"sizeInGiB": 100
},
"executionRoleArn": "arn:aws:iam::123456789012:role/ecsExecutionRole",
"taskRoleArn": "arn:aws:iam::123456789012:role/ecsTaskRole",
"containerDefinitions": [
{
"name": "prefect",
"image": "{{ image }}",
"command": [],
"environment": []
}
]
},
"run_task_kwargs": {
"cluster": "my-cluster",
"launchType": "FARGATE",
"networkConfiguration": {
"awsvpcConfiguration": {
"subnets": [
"subnet-abc"
],
"securityGroups": [
"sg-xyz"
],
"assignPublicIp": "ENABLED"
}
}
}
}
Per-deployment override (Prefect 3.x)
from prefect import flow
@flow
def my_flow():
...
my_flow.from_source(".").deploy(
name="with-ephemeral-storage",
work_pool_name="my-ecs-push",
job_variables={
"task_definition": {
"ephemeralStorage": {"sizeInGiB": 100}
}
},
)
Notes (because AWS loves footnotes):
- Works only for Fargate tasks; valid range is 21–200 GiB. Use platform version 1.4.0+.
- If you need more or persistent storage, mount EFS via `volumes`/`mountPoints` in the task definition instead.
Docs for reference:
- Prefect ECS guide: Prefect 3 ECS integration guide
- AWS ECS ephemeral storage: ECS Fargate task storage and RegisterTaskDefinition API