Claire Herdeman
11/15/2022, 9:11 PMtask_definition_arn
in an ECSTask
block, details belowClaire Herdeman
11/15/2022, 9:11 PMresource "aws_ecs_task_definition" "prefect2Flow" {
family = "prefect2Flow_${var.env}"
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = "512"
memory = "1024"
task_role_arn = aws_iam_role.prefect_flow_ecs_task_role.arn
execution_role_arn = aws_iam_role.prefect_flow_ecs_execution_role.arn
container_definitions = <<TASK_DEFINITION
[
{
"name": "prefect",
"image": "${aws_ecr_repository.integrations.repository_url}:latest",
"environment": [{"name": "PREFECT_DEBUG_MODE", "value": "1"}],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "${aws_cloudwatch_log_group.prefect_flow.name}",
"awslogs-region": "us-east-2",
"awslogs-stream-prefix": "prefect_flow"
}
}
}
]
TASK_DEFINITION
}
Claire Herdeman
11/15/2022, 9:12 PMecs = ECSTask(
task_definition_arn=task,
cluster=cluster,
network_mode="awsvpc",
task_customizations=[
{
"op": "replace",
"path": f"{NETWORK_PATH}/assignPublicIp",
"value": "DISABLED",
},
{
"op": "add",
"path": f"{NETWORK_PATH}/subnets",
"value": get_networking(env, "Subnets"),
},
{
"op": "add",
"path": f"{NETWORK_PATH}/securityGroups",
"value": get_networking(env, "SecurityGroups"),
}
]
)
ecs.save(f"{env}", overwrite=True)
Claire Herdeman
11/15/2022, 9:22 PM"name": "flow"
in my terraform task definition, and then prefect created a second container and prefect
. Change the container name to prefect resolved that issue.
That said, it's still overwriting the custom image I defined and using prefecthq/prefect:2.6.4-python3.10
insteadClaire Herdeman
11/15/2022, 9:37 PMClaire Herdeman
11/15/2022, 9:44 PMimage
param in ECSTask
seems like a likely culprit. However the value is defaulting to a base image even though the image is already defined in the container definition above
"The image to use for the Prefect container in the task. If this value is "
"not null, it will override the value in the task definition. This value "
"defaults to a Prefect base image matching your local versions."
Mason Menges
11/15/2022, 10:31 PMimage=None
will prevent it from doing so.Mason Menges
11/15/2022, 10:31 PMClaire Herdeman
11/16/2022, 3:20 PMimage=None
fixed this one! My one nitpick on the wording of that documentation, right now the behavior is:
⢠Specify image to overwrite the image in the task definition
⢠If you don't specify an image value, it will ALSO overwrite your task definition with a default
⢠Explicitly set image=None to use an image that you've specified
Behavior in point 2 is a bit confusing, I think it would be helpful to explicitly state point 3Mason Menges
11/16/2022, 4:18 PMVishnu Duggirala
11/18/2022, 4:55 PMimage=None
but I get this error 16:14:34.470 | ERROR | prefect.agent - Failed to submit flow run 'ce9f7889-713c-4d53-96d7-56b70190eccd' to infrastructure.
I have no errors when I explicitly state the image.Claire Herdeman
11/18/2022, 5:06 PMVishnu Duggirala
11/18/2022, 5:57 PMVishnu Duggirala
11/21/2022, 8:25 PMContainer.image should not be null or empty
, task-def arn has the imageClaire Herdeman
11/21/2022, 8:40 PMtask_definition_arn
?Vishnu Duggirala
11/21/2022, 8:48 PMVishnu Duggirala
11/21/2022, 8:52 PMClaire Herdeman
11/21/2022, 9:08 PMtask_definition_arn=task_arn
, what is the definition for what task_arn
refers to?
Here is my example. Is the image specified in your container definition? if so, is the name of that container prefect
? I initially had "name": "flow"
and that was actually causing a second container to spin up that wasn't using my settings, setting "name": "prefect"
was important for meVishnu Duggirala
11/21/2022, 10:06 PMprefect
, let me try setting the name as prefect
Vishnu Duggirala
11/21/2022, 10:11 PMClaire Herdeman
11/22/2022, 3:09 PMClaire Herdeman
11/22/2022, 3:11 PMprefect
? I only tried that once I saw two containers running but I could have missed it.Mason Menges
11/22/2022, 5:33 PMClaire Herdeman
11/22/2022, 5:37 PMMason Menges
11/22/2022, 5:37 PMVishnu Duggirala
11/22/2022, 8:43 PM