Madushan Pathirana
10/21/2025, 10:19 AMMarvin
10/21/2025, 10:19 AMMarvin
10/21/2025, 10:27 AMEC2. Do not use FARGATE.
- Or capacity_provider_strategy: If you use capacity providers on your cluster, set a strategy referencing your EC2 provider and omit launch_type entirely. Don’t set both.
- cpu/memory: Pick any values that make sense for your EC2 instances (they are not restricted to the Fargate matrix). For your POC, small defaults like cpu: 256, memory: 512 are fine.
- subnets/security groups: Set them under the run-task network configuration (awsvpc). Example below.
- task_role_arn/execution_role_arn: Provide your task role (app permissions) and the ecsTaskExecutionRole (pulling images, logs, etc.).
Example base job template (edit this in Prefect UI → Work Pools → your ECS pool → Edit base job template). Use either launch_type or capacity_provider_strategy, not both:
{
"region": "ap-southeast-1",
"cluster": "your-ecs-cluster",
"image": "public.ecr.aws/your-image:tag",
"cpu": 256,
"memory": 512,
"launch_type": "EC2",
// If you prefer capacity providers instead of launch_type, remove launch_type and use:
// "capacity_provider_strategy": [
// {"capacityProvider": "EC2", "weight": 1}
// ],
"task_role_arn": "arn:aws:iam::123456789012:role/YourTaskRole",
"execution_role_arn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
"network_configuration": {
"awsvpcConfiguration": {
"subnets": ["subnet-abc123", "subnet-def456"],
"securityGroups": ["sg-0123456789abcdef0"],
"assignPublicIp": "DISABLED"
}
}
// Note: Prefect’s ECS worker will construct the task definition.
// For EC2 it will use awsvpc networking by default if you provide awsvpcConfiguration.
}
A few tips for your setup:
- That Fargate error is a strong signal your pool is currently using Fargate. After switching to EC2 (or to an EC2 capacity provider), the error should disappear.
- Ensure your ECS cluster actually has EC2 capacity (container instances) registered and in ACTIVE state. If you want everything on a single host, that host must be an ECS container instance (ECS agent installed) in the cluster.
- If you previously added Fargate-only fields (e.g., platformVersion, requiresCompatibilities: ["FARGATE"], ephemeralStorage), remove them for EC2.
- You can override cpu/memory per deployment using deployment job_variables if you want different sizes by environment.
Relevant docs:
- Prefect ECS guide (Prefect 3): Prefect ECS guide
- Job variables override (per-deployment): Customize job variables
- AWS: Valid Fargate CPU/memory combos: Fargate CPU/memory
If you share your current work pool base job template (redact ARNs/IDs), I can suggest the exact edits to switch it from Fargate to EC2 for your cluster in ap-southeast-1.