https://prefect.io logo
Title
y

Yaron Levi

04/26/2023, 6:35 PM
Hi šŸ‘‹ When trying to run a Flow on ECS Fargate I get the error: Submission failed. KeyError: ā€œNo class found for dispatch key ā€˜ecs-task’ in registry for type ā€˜Block’.ā€ The whole setup here is an ECS Task running as a work pool and listens to new Runs. The Task was started with the command ā€œprefect worker start --pool test-poolā€ Then I defined a new deployment with this configuration:
prefect deployment build jobs/test1/test1.py:test1 \
--name test1 \
--output jobs/test1/deployment \
--storage-block github/yuvital-prefect-github \
--pool test-pool \
--infra-block ecs-task/ecs-test-block
Meaning when the Work Pool Task poll for a new flow to run, it should also create a new ECS Task for that flow run. Looking at the logs I can see that the work pool do get the new flow to run, but fails with: Submission failed. KeyError: ā€œNo class found for dispatch key ā€˜ecs-task’ in registry for type ā€˜Block’.ā€
z

Zanie

04/26/2023, 6:38 PM
Sounds like
prefect-aws
isn’t installed in your worker’s environment — but note that workers are meant to be used with the
ECSWorker
not the
ECSTask
block
i.e you’d want a work pool of type
ecs
rather than
prefect-agent
y

Yaron Levi

04/26/2023, 6:39 PM
Yes
This this the Task definition of the work pool:
{
  "taskDefinitionArn": "arn:aws:ecs:eu-central-1:759317626178:task-definition/agent-task:8",
  "containerDefinitions": [
    {
      "name": "agent-task",
      "image": "prefecthq/prefect:2-python3.10",
      "cpu": 0,
      "portMappings": [
        {
          "name": "agent-task-80-tcp",
          "containerPort": 80,
          "hostPort": 80,
          "protocol": "tcp",
          "appProtocol": "http"
        }
      ],
      "essential": true,
      "command": [
        "prefect",
        "worker",
        "start",
        "--pool",
        "test-pool"
      ],
      "environment": [
        {
          "name": "PREFECT_API_KEY",
          "value": ""
        },
        {
          "name": "PREFECT_API_URL",
          "value": ""
        },
        {
          "name": "EXTRA_PIP_PACKAGES",
          "value": "prefect-aws s3fs"
        }
      ],
      "environmentFiles": [],
      "mountPoints": [],
      "volumesFrom": [],
      "ulimits": [],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-create-group": "true",
          "awslogs-group": "/ecs/agent-task",
          "awslogs-region": "eu-central-1",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ],
  "family": "agent-task",
  "executionRoleArn": "arn:aws:iam::759317626178:role/ecsTaskExecutionRole",
  "networkMode": "awsvpc",
  "revision": 8,
  "volumes": [],
  "status": "ACTIVE",
  "requiresAttributes": [
    {
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "name": "ecs.capability.execution-role-awslogs"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"
    },
    {
      "name": "ecs.capability.task-eni"
    },
    {
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.29"
    }
  ],
  "placementConstraints": [],
  "compatibilities": [
    "EC2",
    "FARGATE"
  ],
  "requiresCompatibilities": [
    "FARGATE"
  ],
  "cpu": "1024",
  "memory": "3072",
  "runtimePlatform": {
    "cpuArchitecture": "X86_64",
    "operatingSystemFamily": "LINUX"
  },
  "registeredAt": "2023-04-26T17:33:57.713Z",
  "registeredBy": "arn:aws:iam::759317626178:root",
  "tags": []
}
z

Zanie

04/26/2023, 6:40 PM
I don’t think ECS respects
EXTRA_PIP_PACKAGES
because it bypasses our entrypoint
y

Yaron Levi

04/26/2023, 6:40 PM
Let me understand because I am little bit confused (-:
I’ve created a new work pool like so:
CleanShot 2023-04-26 at 21.41.05@2x.jpg
z

Zanie

04/26/2023, 6:41 PM
I think if you add
/opt/prefect/entrypoint.sh
at the start of your commands it’ll install
prefect-aws
like you want
y

Yaron Levi

04/26/2023, 6:41 PM
It runs nice on ECS and listens to new Runs
z

Zanie

04/26/2023, 6:42 PM
I see — I don’t think you can set an infrastructure block when using non-agent work pool types (cc @alex)
y

Yaron Levi

04/26/2023, 6:42 PM
How should I define a deployment so that it will be picked up by this worker pool?
Ah so it’s not possible to create a new ECS Task for each Run?
Meaning when the work pool task polled for a new flow, it should create a new ECS Task.
z

Zanie

04/26/2023, 6:44 PM
We do create a new ECS task for each run
That’s how the ECS worker functions
y

Yaron Levi

04/26/2023, 6:45 PM
Ok, so what I am defining wrong here?
z

Zanie

04/26/2023, 6:45 PM
Workers don’t use infrastructure blocks in the same way that agents do
The work pool defines the infrastructure instead
y

Yaron Levi

04/26/2023, 6:46 PM
ok, so instead of:
prefect deployment build jobs/test1/test1.py:test1 \
--name test1 \
--output jobs/test1/deployment \
--storage-block github/yuvital-prefect-github \
--pool test-pool \
--infra-block ecs-task/ecs-test-block
I should do:
prefect deployment build jobs/test1/test1.py:test1 \
--name test1 \
--output jobs/test1/deployment \
--storage-block github/yuvital-prefect-github \
--pool test-pool
(removing the infra from the deployment?)
a

alex

04/26/2023, 6:47 PM
To create a deployment in an
ecs
work pool, you’ll want to use the new Prefect Projects functionality. Following the project tutorial is a good way to get started: https://docs.prefect.io/latest/tutorials/projects/
y

Yaron Levi

04/26/2023, 6:48 PM
ok got it, and following this method, when the worker pool get a new Run, it will create a new ECS Task for it?
a

alex

04/26/2023, 6:51 PM
Yes, the ECS worker will create a new ECS for each run. Looking at your original error, I’m surprised you got that error for this deployment because workers should not execute flow runs from deployments with a storage block. Do you have any other agents polling this work pool by any chance?
y

Yaron Levi

04/26/2023, 6:52 PM
No. I’ve checked. Nobody else is polling this work pool.
And also looking at the logs of the work pool:
Like you’ve said:
a

alex

04/26/2023, 6:57 PM
Gotcha, that’s a bug. It shouldn’t have executed that flow run. Would you please submit an issue for this?
y

Yaron Levi

04/26/2023, 6:59 PM
Sure I’ll try. I am still trying to wrap my head around all the moving parts.