<@ULVA73B9P> how do in configure placement strater...
# ask-marvin
r
@Marvin how do in configure placement stratergy for my ecs work pool / deployment?
m
thought for 508.5 seconds
Great question! Short answer: placement strategy is not currently exposed as a first-class field in Prefect 3’s ECS worker template. You can use capacity provider strategy, and you can enforce placement constraints via the task definition. Placement strategy (the RunTask parameter) is only available on EC2, not on Fargate. A few concrete options and examples: 1) Check what the ECS worker supports by default Run this to see the default ECS base job template:
Copy code
prefect work-pool get-default-base-job-template --type ecs
You’ll see
task_run_request
fields like
cluster
,
overrides
,
taskDefinition
, and
capacityProviderStrategy
, but not
placementStrategy
or
placementConstraints
. Docs: - Prefect ECS guide - Customize job variables - AWS RunTask params: RunTask API - AWS TaskDefinition placement constraints: TaskDefinitionPlacementConstraint 2) If you need EC2-style placement constraints You can bake placement constraints into the task definition (supported by AWS). Add to the work pool’s base job template under
task_definition
or use a pre-created task definition ARN: Example base job template snippet (work pool):
Copy code
{
  "job_configuration": {
    "task_definition": {
      "family": "my-family",
      "cpu": "512",
      "memory": "1024",
      "containerDefinitions": [
        {
          "name": "{{ container_name }}",
          "image": "{{ image }}"
        }
      ],
      "placementConstraints": [
        {
          "type": "memberOf",
          "expression": "attribute:ecs.instance-type =~ t3.*"
        }
      ]
    },
    "task_run_request": {
      "launchType": "EC2",
      "cluster": "my-cluster"
    }
  }
}
Or reference a pre-created task definition that already has constraints:
Copy code
"task_run_request": {
  "taskDefinition": "arn:aws:ecs:...:task-definition/my-family:42",
  "launchType": "EC2",
  "cluster": "my-cluster"
}
3) Capacity provider strategy (supported) If your goal is to steer tasks to specific capacity, you can use
capacityProviderStrategy
(works with both EC2 capacity providers and FARGATE/FARGATE_SPOT):
Copy code
"task_run_request": {
  "cluster": "my-cluster",
  "capacityProviderStrategy": [
    {"capacityProvider": "FARGATE_SPOT", "weight": 1}
  ]
}
4) What about placementStrategy (spread/binpack) on EC2? - Prefect’s default ECS template does not expose
placementStrategy
under
task_run_request
. Even if you add it to job variables, some versions of prefect-aws validate/shape this payload and may reject unknown keys. - If you try to put:
Copy code
"task_run_request": {
  "launchType": "EC2",
  "cluster": "my-cluster",
  "placementStrategy": [
    {"type": "spread", "field": "attribute:ecs.availability-zone"},
    {"type": "binpack", "field": "cpu"}
  ]
}
and get an “extra fields not permitted” error, that means your version does not pass these through. Workarounds if you need true placementStrategy: - Use capacity providers and/or constraints in the task definition to approximate placement. - If you absolutely must use RunTask placementStrategy, you’d need a small customization (e.g., a custom worker that adds these fields to the boto3 call). See Developing a custom worker. 5) How to apply via deployment job variables You can override the pool’s template at deploy-time using
job_variables
in your `prefect.yaml`:
Copy code
deployments:
- name: my-ecs-deployment
  work_pool:
    name: my-ecs-pool
  job_variables:
    task_run_request:
      launchType: EC2
      cluster: my-cluster
    task_definition:
      placementConstraints:
        - type: memberOf
          expression: attribute:ecs.instance-type =~ t3.*
Questions to make sure we’re steering you right: - Are you running on Fargate or EC2? (Placement strategies/constraints apply to EC2; Fargate does not support them.) - Do you prefer to reference a pre-existing task definition (with constraints) or let Prefect register one per run? If you share those details, I can provide an exactly-tailored snippet you can paste into your work pool or deployment.
r
@Marvin since placement strategy is a parameter to ecs runTask, why isnt this part of default implementation?
m
thought for 73.5 seconds
Great question. A few reasons it isn’t in the default ECS worker template: - It’s EC2-only.
placementStrategy
is a RunTask parameter that AWS does not support on Fargate. Since Prefect’s ECS worker supports both Fargate and EC2, the default template sticks to fields that won’t error on Fargate. - Keep the job template small and predictable. Prefect’s ECS worker passes a curated set of RunTask fields (cluster, overrides, taskDefinition, capacityProviderStrategy, etc.). Exposing every RunTask option would bloat the schema and increase validation/compat risk. - Capacity providers are the preferred, broadly applicable control. The template includes
capacityProviderStrategy
(works for EC2 capacity providers and FARGATE/FARGATE_SPOT), which covers many placement needs without EC2-only knobs. - Defaults work for most users. ECS’s scheduler does a reasonable job without explicit placement strategies, so the default avoids adding a field that can fail or confuse in mixed (Fargate/EC2) setups. If you need it on EC2, here are your options today: - Use capacity providers to steer placement:
Copy code
"task_run_request": {
  "cluster": "my-cluster",
  "capacityProviderStrategy": [
    {"capacityProvider": "EC2ProviderA", "weight": 1},
    {"capacityProvider": "EC2ProviderB", "weight": 2}
  ]
}
- Enforce constraints via the task definition (EC2-only). While
placementStrategy
is RunTask-only, you can still filter with
placementConstraints
inside the task definition:
Copy code
"task_definition": {
  "family": "my-family",
  "cpu": "512",
  "memory": "1024",
  "containerDefinitions": [
    {"name": "{{ container_name }}", "image": "{{ image }}"}
  ],
  "placementConstraints": [
    {"type": "memberOf", "expression": "attribute:ecs.instance-type =~ t3.*"}
  ]
}
- Try adding
placementStrategy
in your work pool’s base job template (EC2 only). Example:
Copy code
"task_run_request": {
  "launchType": "EC2",
  "cluster": "my-cluster",
  "placementStrategy": [
    {"type": "spread", "field": "attribute:ecs.availability-zone"},
    {"type": "binpack", "field": "cpu"}
  ]
}
Note: depending on your prefect-aws version, extra keys may be ignored or rejected by the worker’s schema. If it doesn’t pass through, we can open a feature request to add first-class support. Links: - Prefect ECS guide - Customize job variables - Developing a custom worker - AWS RunTask API If you’re on EC2 and specifically need `spread`/`binpack`, want me to check whether your current version will pass
placementStrategy
through if we add it to your base job template? If you share EC2 vs Fargate and your prefect/prefect-aws versions, I can provide an exact template you can drop in.