Ravish Mallya
09/18/2025, 3:29 PMMarvin
09/18/2025, 3:29 PMMarvin
09/18/2025, 3:38 PMprefect work-pool get-default-base-job-template --type ecs
You’ll see task_run_request
fields like cluster
, overrides
, taskDefinition
, and capacityProviderStrategy
, but not placementStrategy
or placementConstraints
.
Docs:
- Prefect ECS guide
- Customize job variables
- AWS RunTask params: RunTask API
- AWS TaskDefinition placement constraints: TaskDefinitionPlacementConstraint
2) If you need EC2-style placement constraints
You can bake placement constraints into the task definition (supported by AWS). Add to the work pool’s base job template under task_definition
or use a pre-created task definition ARN:
Example base job template snippet (work pool):
{
"job_configuration": {
"task_definition": {
"family": "my-family",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{
"name": "{{ container_name }}",
"image": "{{ image }}"
}
],
"placementConstraints": [
{
"type": "memberOf",
"expression": "attribute:ecs.instance-type =~ t3.*"
}
]
},
"task_run_request": {
"launchType": "EC2",
"cluster": "my-cluster"
}
}
}
Or reference a pre-created task definition that already has constraints:
"task_run_request": {
"taskDefinition": "arn:aws:ecs:...:task-definition/my-family:42",
"launchType": "EC2",
"cluster": "my-cluster"
}
3) Capacity provider strategy (supported)
If your goal is to steer tasks to specific capacity, you can use capacityProviderStrategy
(works with both EC2 capacity providers and FARGATE/FARGATE_SPOT):
"task_run_request": {
"cluster": "my-cluster",
"capacityProviderStrategy": [
{"capacityProvider": "FARGATE_SPOT", "weight": 1}
]
}
4) What about placementStrategy (spread/binpack) on EC2?
- Prefect’s default ECS template does not expose placementStrategy
under task_run_request
. Even if you add it to job variables, some versions of prefect-aws validate/shape this payload and may reject unknown keys.
- If you try to put:
"task_run_request": {
"launchType": "EC2",
"cluster": "my-cluster",
"placementStrategy": [
{"type": "spread", "field": "attribute:ecs.availability-zone"},
{"type": "binpack", "field": "cpu"}
]
}
and get an “extra fields not permitted” error, that means your version does not pass these through.
Workarounds if you need true placementStrategy:
- Use capacity providers and/or constraints in the task definition to approximate placement.
- If you absolutely must use RunTask placementStrategy, you’d need a small customization (e.g., a custom worker that adds these fields to the boto3 call). See Developing a custom worker.
5) How to apply via deployment job variables
You can override the pool’s template at deploy-time using job_variables
in your `prefect.yaml`:
deployments:
- name: my-ecs-deployment
work_pool:
name: my-ecs-pool
job_variables:
task_run_request:
launchType: EC2
cluster: my-cluster
task_definition:
placementConstraints:
- type: memberOf
expression: attribute:ecs.instance-type =~ t3.*
Marvin
09/18/2025, 3:38 PMRavish Mallya
09/18/2025, 3:46 PMMarvin
09/18/2025, 3:46 PMMarvin
09/18/2025, 3:47 PMplacementStrategy
is a RunTask parameter that AWS does not support on Fargate. Since Prefect’s ECS worker supports both Fargate and EC2, the default template sticks to fields that won’t error on Fargate.
- Keep the job template small and predictable. Prefect’s ECS worker passes a curated set of RunTask fields (cluster, overrides, taskDefinition, capacityProviderStrategy, etc.). Exposing every RunTask option would bloat the schema and increase validation/compat risk.
- Capacity providers are the preferred, broadly applicable control. The template includes capacityProviderStrategy
(works for EC2 capacity providers and FARGATE/FARGATE_SPOT), which covers many placement needs without EC2-only knobs.
- Defaults work for most users. ECS’s scheduler does a reasonable job without explicit placement strategies, so the default avoids adding a field that can fail or confuse in mixed (Fargate/EC2) setups.
If you need it on EC2, here are your options today:
- Use capacity providers to steer placement:
"task_run_request": {
"cluster": "my-cluster",
"capacityProviderStrategy": [
{"capacityProvider": "EC2ProviderA", "weight": 1},
{"capacityProvider": "EC2ProviderB", "weight": 2}
]
}
- Enforce constraints via the task definition (EC2-only). While placementStrategy
is RunTask-only, you can still filter with placementConstraints
inside the task definition:
"task_definition": {
"family": "my-family",
"cpu": "512",
"memory": "1024",
"containerDefinitions": [
{"name": "{{ container_name }}", "image": "{{ image }}"}
],
"placementConstraints": [
{"type": "memberOf", "expression": "attribute:ecs.instance-type =~ t3.*"}
]
}
- Try adding placementStrategy
in your work pool’s base job template (EC2 only). Example:
"task_run_request": {
"launchType": "EC2",
"cluster": "my-cluster",
"placementStrategy": [
{"type": "spread", "field": "attribute:ecs.availability-zone"},
{"type": "binpack", "field": "cpu"}
]
}
Note: depending on your prefect-aws version, extra keys may be ignored or rejected by the worker’s schema. If it doesn’t pass through, we can open a feature request to add first-class support.
Links:
- Prefect ECS guide
- Customize job variables
- Developing a custom worker
- AWS RunTask API
If you’re on EC2 and specifically need `spread`/`binpack`, want me to check whether your current version will pass placementStrategy
through if we add it to your base job template? If you share EC2 vs Fargate and your prefect/prefect-aws versions, I can provide an exact template you can drop in.