Madushan Pathirana
10/30/2025, 4:47 AMMarvin
10/30/2025, 4:47 AMMarvin
10/30/2025, 4:56 AMjob_variables on the deployment to point at a US‑region ECS cluster (use a full cluster ARN; the ARN includes the region).
- The worker can stay on your EC2 in ap-southeast-1; it will launch tasks in the US via AWS APIs as long as its IAM creds are valid in that region.
Example:
from prefect import flow
@flow
def my_flow():
...
# Deploy this flow to run in the US region
my_flow.deploy(
name="my-flow-us",
work_pool_name="ecs-pool",
job_variables={
"cluster": "arn:aws:ecs:us-east-1:123456789012:cluster/my-us-cluster",
"launch_type": "EC2",
"subnet_ids": ["subnet-aaaa", "subnet-bbbb"],
"security_group_ids": ["sg-cccccccc"],
"assign_public_ip": False,
# optional, small POC defaults
"cpu": 256,
"memory": 512
},
)
2) Create a second work pool per region
- Make an ecs-us pool whose base job variables default to your US cluster/subnets/SGs.
- Run a second worker on the same EC2 host that polls the US pool.
- This gives clear separation and per‑pool concurrency limits, at the cost of an extra worker process.
Notes and prerequisites:
- Ensure you have an ECS EC2 cluster with capacity in the target US region (EC2 instances joined to the cluster, correct AMI/ECS agent, subnets/SGs).
- IAM: the worker’s AWS credentials must allow ECS operations in that US region/account.
- Networking: subnet_ids and security_group_ids must be from the US VPC. If these flows need to reach your RDS in Singapore, cross‑region latency and SG rules may apply.
- You’re using launch type EC2 (not Fargate), which is supported—keep launch_type: "EC2".
Docs for more detail:
- ECS worker guide
- Customize job variables
Want me to sketch the exact job_variables for your US cluster if you share the region (e.g., us-east-1 vs us-west-2), cluster ARN, and VPC subnet/SG IDs?