Madushan Pathirana
09/15/2025, 2:02 AMMarvin
09/15/2025, 2:02 AMMarvin
09/15/2025, 2:07 AMdeployments:
- name: my-flow-small
entrypoint: flows/my_flow.py:my_flow
work_pool:
name: my-ecs-pool
job_variables:
cluster: arn:aws:ecs:ap-southeast-1:123456789012:cluster/my-ecs-cluster
launch_type: EC2
cpu: 256 # 0.25 vCPU
memory: 512 # MiB
subnets: ["subnet-abc123"]
security_groups: ["sg-abc123"]
assign_public_ip: false
task_role_arn: arn:aws:iam::123456789012:role/my-ecs-task-role
execution_role_arn: arn:aws:iam::123456789012:role/my-ecs-exec-role
# container overrides
env:
PREFECT_API_URL: http://<your-ec2-ip>:4200/api
command: [] # leave empty to use flow's default
- name: my-flow-large
entrypoint: flows/my_flow.py:my_flow
work_pool:
name: my-ecs-pool
job_variables:
cluster: arn:aws:ecs:ap-southeast-1:123456789012:cluster/my-ecs-cluster
launch_type: EC2
cpu: 1024 # 1 vCPU
memory: 2048 # MiB
subnets: ["subnet-abc123"]
security_groups: ["sg-abc123"]
assign_public_ip: false
task_role_arn: arn:aws:iam::123456789012:role/my-ecs-task-role
execution_role_arn: arn:aws:iam::123456789012:role/my-ecs-exec-role
env:
PREFECT_API_URL: http://<your-ec2-ip>:4200/api
command: []
Start multiple workers for the same pool (optionally pinned to different queues):
prefect worker start -p my-ecs-pool -n worker-small --work-queue small
prefect worker start -p my-ecs-pool -n worker-large --work-queue large
Notes for your setup (ECS EC2 on a single host)
- Ensure your requested cpu/memory combinations are supported by your EC2 container instance and ECS task definition constraints.
- If you’re using capacity providers for EC2, set a capacity_provider_strategy instead of launch_type, otherwise keep launch_type: EC2.
- For clean separation (e.g., small vs large, or dev/stage/prod), use work queues and target workers to specific queues.
Docs
- ECS guide: Prefect ECS guide
- Customizing job variables: Customize job variables
- Variables concept (for templating and overrides): Variables in Prefect 3
If you want, I can sketch a pool base job template JSON for ECS EC2 as a starting point and show how to wire queues for small/large.Madushan Pathirana
09/15/2025, 2:39 AMMarvin
09/15/2025, 2:39 AMresearch_prefect_topic
📊 Tools used:
• check_cli_command (2x)
• research_prefect_topic (1x)