Anton L.
10/26/2025, 8:36 PMMarvin
10/26/2025, 8:36 PMMarvin
10/26/2025, 8:47 PMjob_configuration.task_definition.containerDefinitions. In Prefect 3.x, you can add privileged and linuxParameters.sharedMemorySize to the container definition so every flow run launched by that pool inherits those settings.
Important notes:
- Privileged mode and shared memory sizing are only supported on ECS EC2 launch type. Fargate does not support privileged or linuxParameters.sharedMemorySize. If your pool uses Fargate, you’ll need to switch to EC2 (via capacity providers or launchType: EC2) for these to work.
Example base job template edit (showing only the relevant sections). Add the highlighted fields under `containerDefinitions`:
{
"job_configuration": {
"task_definition": {
"containerDefinitions": [
{
"image": "{{ image }}",
"name": "{{ container_name }}",
"privileged": true,
"linuxParameters": {
"sharedMemorySize": 4096
}
}
],
"cpu": "{{ cpu }}",
"family": "{{ family }}",
"memory": "{{ memory }}",
"executionRoleArn": "{{ execution_role_arn }}"
},
"task_run_request": {
"launchType": "{{ launch_type }}",
"cluster": "{{ cluster }}",
"overrides": {
"containerOverrides": [
{
"name": "{{ container_name }}",
"command": "{{ command }}",
"environment": "{{ env }}",
"cpu": "{{ cpu }}",
"memory": "{{ memory }}"
}
],
"cpu": "{{ cpu }}",
"memory": "{{ memory }}",
"taskRoleArn": "{{ task_role_arn }}"
},
"tags": "{{ labels }}",
"taskDefinition": "{{ task_definition_arn }}",
"capacityProviderStrategy": "{{ capacity_provider_strategy }}"
},
"container_name": "{{ container_name }}",
"cluster": "{{ cluster }}"
}
}
How to apply this:
1) Fetch the current default template to a file:
prefect work-pool get-default-base-job-template --type ecs > ecs-template.json
2) Edit ecs-template.json to add privileged and linuxParameters.sharedMemorySize as shown above.
3) Update your work pool:
prefect work-pool update "<your-pool-name>" --base-job-template ecs-template.json
Or do the same edit in the Prefect UI: Work Pools > your ECS pool > Base job template > JSON editor.
A couple of gotchas:
- If you set task_run_request.launchType to FARGATE or use a Fargate capacity provider, AWS will reject privileged and sharedMemorySize. Use EC2 capacity instead.
- If you provide a task_definition_arn at deployment time, that ARN is used as-is and will override the template-synthesized task definition. In that case, put these settings into the task definition you reference.
References
- Prefect ECS guide: Prefect 3 ECS guide
- Work pool CLI (for base job templates): prefect work-pool CLI docs
- AWS ECS ContainerDefinition and linuxParameters:
- ContainerDefinition API
- LinuxParameters API
- Fargate limitations (no privileged / shm sizing): Fargate task definition support
Quick sanity check: Are you using ECS with EC2 capacity (not Fargate) on this work pool? If you’re on Fargate, I can suggest alternatives or help you switch to EC2 capacity providers.