I'm having an issue where the task definition that is created from the agent seems to not be correct. The task definition for the flow looks like this: (see screenshot below) But, when I am running my agent, I am using

prefect agent ecs start --cluster arn:aws:ecs:us-west-2:xxx:cluster/staging-cluster --label staging --task-role-arn arn:aws:iam::xxx:role/staging-prefect-agent-flow --log-level DEBUG --launch-type FARGATE

. So I would expect the compatibility to be

FARGATE

It seems this is the cause of the error I am getting when I submit my flow to the agent. The error is:

[2020-12-18 00:34:44,559] ERROR - agent | Error while deploying flow: InvalidParameterException('An error occurred (InvalidParameterException) when calling the RunTask operation: Task definition does not support launch_type FARGATE.')

Has anyone seen something like this before?

Maybe it is related to the compatibility. From the screenshot I see only EC2, but not Fargate

I am having exactly the same issue @Marc Lipoff -- did you manage to get it working?

Hi @Kieran I did. Here is my terraform resource. Hope this helps.

resource "aws_ecs_task_definition" "prefect-agent-runner" {
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  family                   = local.service_name
  cpu                = 1024
  memory             = 3072
  task_role_arn      = aws_iam_role.task-role.arn
  execution_role_arn = data.aws_iam_role.ecs-task-execution-role.arn
  container_definitions = jsonencode([
    {
        "entryPoint" = [
            "sh",
            "-c"
        ],
        "command": [
            join(" ", ["sh", "-c",
        <<COMMANDBLOCK
'pip install prefect[aws];
cat <<EOF > ./task_definition.yaml;
networkMode: awsvpc
cpu: 1024
memory: 2048
executionRoleArn: arn:aws:iam::xxx:role/${terraform.workspace}-ecs-instance-role
requiresCompatibilities:
  - FARGATE
containerDefinitions:
  - name: flow
EOF
cat <<EOF > ./run_task_kwargs.yaml;
networkConfiguration:
    awsvpcConfiguration:
        assignPublicIp: DISABLED
        securityGroups:
            - ${aws_security_group.flows-task.id}
        subnets:
            - ${data.aws_subnet.private[0].id}
            - ${data.aws_subnet.private[1].id}
EOF
prefect agent ecs start --cluster arn:aws:ecs:us-west-2:xxx:cluster/${terraform.workspace}-cluster --label ${terraform.workspace} --task-role-arn ${aws_iam_role.flow-task-role.arn} --log-level ${local.agent_log_level} --task-definition ./task_definition.yaml --run-task-kwargs ./run_task_kwargs.yaml'
          COMMANDBLOCK
        ])],
        "environment": [
            {"name": "VERSION_INFO", "value": "v0"},
            {"name": "BUILD_PREFECT_RUNNER", "value": "-"}
        ],
        "secrets": [
          {
            "name": "PREFECT__CLOUD__AGENT__AUTH_TOKEN",
            "valueFrom": "arn:aws:secretsmanager:us-west-2:xxx:secret:PREFECT__CLOUD__AGENT__AUTH_TOKEN"
          }
        ],
        "essential": true,
        "image": "prefecthq/prefect:0.14.0",
        "name": "${local.service_name}",
        "logConfiguration": {
            "logDriver": "awslogs",
            "options": {
                "awslogs-group": "/fargate/service/${local.service_name}",
                "awslogs-region": "us-west-2",
                "awslogs-stream-prefix": "ecs"
            }
        }
    }
  ])
}

Interesting, so you used Terraform to see the default task definition and declare the FARGATE launch type. I assumed that the agent would do that after selecting --launch-type on start. Thanks @Marc Lipoff I will give that a go as we are also using Terraform to provision infrastructure.

👍 Ya, we use terraform for everything too. Ya, for some reason, when not specify the run_task_kwargs and task_definition, prefect chooses incorrect values. This just forces "good" values.