Wolfgang Dieterich
08/20/2025, 5:05 PMdeployments:
- name: my_ecs_deployment_dev
entrypoint: prefect/my_flow.py:my_flow
job_variables:
image: "{{ $image_tag }}"
work_pool:
name: *dev_work_pool
image: "{{ $image_tag }}"
However when I deploy this and schedule a run, the image specified in the deployment is not used. when i look in the prefect ui, there are no job variables and my deployment doesn't reference my image name anywhere.Marvin
08/20/2025, 5:05 PMMarvin
08/20/2025, 5:11 PMwork_pool.image
isn’t a valid key. Your top-level job_variables
is also ignored. That’s why the UI shows no job variables and your image isn’t used.
Here’s the corrected structure for an ECS work pool deployment:
# optional if you want templating
variables:
image_tag: "<http://123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:latest|123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:latest>"
deployments:
- name: my_ecs_deployment_dev
entrypoint: prefect/my_flow.py:my_flow
work_pool:
name: dev_work_pool
job_variables:
image: "{{ $image_tag }}"
Notes:
- job_variables
must be nested under work_pool
.
- image
is the right key for ECS workers.
- {{ $image_tag }}
will only render if you define image_tag
in the top-level variables:
block (as above) or via your project variables. Otherwise it renders to nothing, which is… unhelpful.
After updating, redeploy from your project root:
prefect deploy
Then check the deployment in the UI — you should see the job variables (including the image) on the deployment page. If you still don’t see them:
- Confirm the work pool is of type ECS and your worker is running against that pool.
- Make sure you deployed from the same project directory containing this prefect.yaml.
- Verify the variable actually resolves (try hardcoding the image once to confirm the path).
Docs, in case you like reading specifications as much as I do:
- Customize job variables: Customize job variables
- prefect.yaml reference: prefect.yaml reference
- ECS guide: AWS ECS guide
If you share how you’re setting image_tag
and the exact work pool type/name, I’ll sanity-check the rest. I live for this. Obviously.Wolfgang Dieterich
08/20/2025, 5:16 PMWolfgang Dieterich
08/20/2025, 5:17 PMMarvin
08/20/2025, 5:17 PMMarvin
08/20/2025, 5:20 PM{
"executionRoleArn": "{{ execution_role_arn }}",
"taskRoleArn": "{{ task_role_arn }}",
"containerDefinitions": [
{
"image": "{{ image }}",
...
}
],
"networkConfiguration": {
"awsvpcConfiguration": {
"subnets": {{ subnets }},
"securityGroups": {{ security_groups }},
"assignPublicIp": "{{ assign_public_ip }}"
}
},
"cpu": "{{ cpu }}",
"memory": "{{ memory }}",
"cluster": "{{ cluster }}"
}
If executionRoleArn isn’t present in the template, Prefect can’t set it and ECS will throw exactly the error you’re seeing.
3) Pass the variables in your deployment
variables:
image_tag: "<http://123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:latest|123456789012.dkr.ecr.us-east-1.amazonaws.com/my-image:latest>"
deployments:
- name: my_ecs_deployment_dev
entrypoint: prefect/my_flow.py:my_flow
work_pool:
name: dev_work_pool
job_variables:
image: "{{ $image_tag }}"
execution_role_arn: "arn:aws:iam::123456789012:role/ecsTaskExecutionRole"
task_role_arn: "arn:aws:iam::123456789012:role/my-ecs-task-role"
cluster: "my-ecs-cluster"
cpu: "512"
memory: "1024"
subnets: ["subnet-abc", "subnet-def"]
security_groups: ["sg-123456"]
assign_public_ip: "ENABLED"
Then redeploy:
prefect deploy
If you’re unsure of the exact variable names, open your work pool’s base job template — those placeholders determine the correct keys. Prefect doesn’t invent them at runtime.
If you share your ECS work pool base job template (or a redacted version), I’ll confirm the exact keys you should set. Also confirm your worker’s AWS credentials have iam:PassRole for both roles; without that, ECS can’t use them even if they’re set.