Thanks for the details — this error usually happens when ECS receives a run_task request with launchType=EC2 but there are no registered container instances yet. If your ASG is at desired=0 and you intend to rely on an EC2 Capacity Provider to scale up, ECS must receive a capacityProviderStrategy (and no launchType) so it can trigger managed scaling. Two likely issues to fix: 1) Key name inside capacity provider strategy - In the AWS API, each item requires the key capacityProvider (camelCase), not capacity_provider. - If the key is wrong, Prefect won’t detect a valid strategy and will still send launchType=EC2 — which triggers your exact “No Container Instances” error. Correct example in prefect.yaml (Prefect 3.x), under your deployment’s work_pool.job_variables:

deployments:
- name: my-deployment
  work_pool:
    name: my-ecs-pool
    job_variables:
      cluster: prefect-ecs-ec2-cluster
      # Important: use capacityProvider (camelCase) in the list items
      capacity_provider_strategy:
        - capacityProvider: Infra-ECS-Cluster-prefect-ecs-ec2-cluster-e6bf397c-AsgCapacityProvider-RKpcHqwBUE59
          weight: 1
          base: 0
      # Do NOT set launch_type when using capacity providers
      # launch_type: null

2) Ensure the capacity provider can actually scale from 0 - The capacity provider must be attached to your ECS cluster and configured with Managed Scaling enabled, pointing to the ASG you mentioned. - When ECS receives a run_task with a capacityProviderStrategy, it will use the provider’s managed scaling to scale the ASG up from 0. If managed scaling isn’t enabled or misconfigured, ECS cannot scale the ASG and run_task may fail. - If you want this hands-off, consider setting the cluster’s default capacity provider strategy in ECS so you don’t need to pass it from Prefect. Why you saw an immediate failure - If Prefect did not pass a valid capacityProviderStrategy (due to the wrong key), it sent launchType=EC2. With desired=0 and no instances registered, ECS immediately returns “No Container Instances”, and Prefect surfaces that error. Recommendations - Update the key to capacityProvider as shown above and make sure you are not setting launch_type. - Verify the capacity provider is attached to the cluster and Managed Scaling is enabled for the ASG. - Optional: Increase the “Task start timeout” in your ECS work pool settings (in the Prefect UI) to allow enough time for ASG scale-up on cold starts. - Alternatively, configure a default capacity provider strategy at the ECS cluster level so Prefect doesn’t need to pass it at all. Helpful AWS docs - Capacity providers overview: AWS: Cluster capacity providers - RunTask and capacityProviderStrategy: AWS API: RunTask and CapacityProviderStrategyItem If you can share: - The exact section of your prefect.yaml under work_pool.job_variables - Whether the cluster shows your capacity provider attached with Managed Scaling enabled - Whether you have any launch_type set …I can sanity-check the configuration and get you unblocked quickly.

I tried using camel case for

capacity_provider_strategy

and

capacity_provider

@Marvin

{
  "cpu": 49152,
  "env": {
    "AWS_REGION": "us-east-1",
    "SEC_API_KEY": "XYZ",
    "SNOWFLAKE_PW": "XYZ",
    "AWS_ACCESS_KEY_ID": "XYZ",
    "AWS_SECRET_ACCESS_KEY": "XYZ"
  },
  "image": "etl:0.8.0",
  "memory": 380673,
  "cluster": "prefect-ecs-ec2-cluster",
  "aws_credentials": {
    "region_name": "us-east-1",
    "aws_access_key_id": "XYZ",
    "aws_secret_access_key": "XYZ",
  },
  "execution_role_arn": "arn:aws:iam::XYZ:role/PrefectEcsTaskExecutionRole",
  "cloudwatch_logs_options": {
    "awslogs-group": "/ecs/prefect-ecs-ec2-etl",
    "awslogs-region": "us-east-1",
    "awslogs-stream-prefix": "etl"
  },
  "configure_cloudwatch_logs": true,
  "capacity_provider_strategy": [
    {
      "base": 0,
      "weight": 1,
      "capacity_provider": "Infra-ECS-Cluster-prefect-ecs-ec2-cluster-e6bf397c-AsgCapacityProvider-RKpcHqwBUE59"
    }
  ]
}

The capacity is set and defaulted. [11:14 PM] I tried using camel case for

capacity_provider_strategy

and

capacity_provider

Thanks for sharing the config — this helps. Key points for ECS capacity providers in Prefect 3.x: - The top-level field must be

capacity_provider_strategy

(snake_case). - Each entry inside the list must use camelCase keys:

capacityProvider

,

weight

,

base

. - When

capacity_provider_strategy

is present, Prefect will remove

launchType

from the ECS RunTask request (required by AWS). If you rely on a cluster default capacity provider strategy and do not set one in Prefect, Prefect will default

launchType

to

FARGATE

, which will override your cluster default. In other words: if you want ECS capacity providers to be used, you must set

capacity_provider_strategy

in your job variables. Given your snippet, here’s the corrected shape (note the inner camelCase key and keeping the top-level as snake_case):

{
  "cpu": 49152,
  "env": {
    "AWS_REGION": "us-east-1",
    "SEC_API_KEY": "XYZ",
    "SNOWFLAKE_PW": "XYZ",
    "AWS_ACCESS_KEY_ID": "XYZ",
    "AWS_SECRET_ACCESS_KEY": "XYZ"
  },
  "image": "etl:0.8.0",
  "memory": 380673,
  "cluster": "prefect-ecs-ec2-cluster",
  "aws_credentials": {
    "region_name": "us-east-1",
    "aws_access_key_id": "XYZ",
    "aws_secret_access_key": "XYZ"
  },
  "execution_role_arn": "arn:aws:iam::XYZ:role/PrefectEcsTaskExecutionRole",
  "cloudwatch_logs_options": {
    "awslogs-group": "/ecs/prefect-ecs-ec2-etl",
    "awslogs-region": "us-east-1",
    "awslogs-stream-prefix": "etl"
  },
  "configure_cloudwatch_logs": true,
  "capacity_provider_strategy": [
    {
      "capacityProvider": "Infra-ECS-Cluster-prefect-ecs-ec2-cluster-e6bf397c-AsgCapacityProvider-RKpcHqwBUE59",
      "weight": 1,
      "base": 0
    }
  ]
}

A few gotchas to double-check: - Do not camelCase the top-level key:

capacity_provider_strategy

must be snake_case. Using

capacityProviderStrategy

at the top level will be ignored. - The inner provider key must be

capacityProvider

(camelCase). Using

capacity_provider

will be ignored by AWS. - If your task definition container is not named

prefect

, set

container_name

in your job variables so the CPU/memory overrides apply to the correct container. - For EC2/capacity providers, Prefect applies

cpu

and

memory

at the container level. CPU is in CPU units (1024 = 1 vCPU); memory is in MiB. Ensure these values fit your instance sizes. Very large values (e.g., 48 vCPU / ~371 GiB) must match available capacity. - If your task definition uses

awsvpc

network mode on EC2, provide

network_configuration

(subnets/security groups). Bridge mode requires none. If you still see it ignoring your capacity provider, can you share: - The exact job variables you used when you tried camel case - The ECS worker logs around task submission (there should be a warning like “Removing launchType from task run request” when

capacity_provider_strategy

is present) - Your task definition’s container names (to confirm

container_name

alignment) Docs and references: - Prefect ECS worker docs: https://docs.prefect.io/latest/integrations/prefect-aws/ecs_worker/ - AWS RunTask API (capacityProviderStrategy vs launchType): https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RunTask.html

Short answer: not under

job_variables

. -

requiresCompatibilities

is an AWS ECS task definition field. In Prefect 3.x you set it in the ECS work pool’s base job template under

job_configuration.task_definition

, not in a deployment’s

prefect.yaml

. -

compatibilities

is a read-only field returned by ECS after registration; you don’t set it at all. What you can do in prefect.yaml: - In a deployment’s

prefect.yaml

, you can only set

work_pool.job_variables

(e.g.,

image

,

cpu

,

memory

,

launch_type

). When

launch_type

is

FARGATE

, the ECS worker will automatically ensure

requiresCompatibilities: ["FARGATE"]

is present. If you really need to hard-set it, edit the ECS work pool’s base job template (in the UI). Example snippet for the work pool template:

{
  "job_configuration": {
    "launch_type": "FARGATE",
    "task_definition": {
      "family": "prefect",
      "networkMode": "awsvpc",
      "requiresCompatibilities": [
        "FARGATE"
      ],
      "cpu": "{{ cpu }}",
      "memory": "{{ memory }}",
      "containerDefinitions": [
        {
          "name": "{{ container_name }}",
          "image": "{{ image }}"
        }
      ]
    }
  }
}

Then your deployment’s prefect.yaml would only set job variables:

deployments:
- name: my-flow
  entrypoint: flow.py:flow
  work_pool:
    name: my-ecs-pool
    job_variables:
      image: myrepo/myimage:tag
      cpu: 1024
      memory: 2048
      launch_type: FARGATE

If you’re not using an ECS work pool, these fields aren’t applicable. Are you targeting ECS Fargate or EC2? I can tailor the exact template for your case.