Hi wave When trying to run a Flow on ECS Fargate I get the e

Question

Hi wave When trying to run a Flow on ECS Fargate I get the error Submission failed KeyError No class found for dispatch key ecs task in registry for type Block The whole setup here is an ECS Task running as a work pool and listens to new Runs The Task was started with the command prefect worker start pool test pool Then I defined a new deployment with this configuration ```prefect deployment build jobs test1 test1 py test1 name test1 output jobs test1 deployment storage block github yuvital prefect github pool test pool infra block ecs task ecs test block``` Meaning when the Work Pool Task poll for a new flow to run it should also create a new ECS Task for that flow run Looking at the logs I can see that the work pool do get the new flow to run but fails with Submission failed KeyError No class found for dispatch key ecs task in registry for type Block

Zanie · Answer

Sounds like `prefect aws` isn t installed in your worker s environment but note that workers are meant to be used with the `ECSWorker` not the `ECSTask` block

Zanie · Answer

i e you d want a work pool of type `ecs` rather than `prefect agent`

Yaron Levi · Answer

Yes

Yaron Levi · Answer

This this the Task definition of the work pool

Yaron Levi · Answer

``` taskDefinitionArn arn aws ecs eu central 1 759317626178 task definition agent task 8 containerDefinitions name agent task image prefecthq prefect 2 python3 10 cpu 0 portMappings name agent task 80 tcp containerPort 80 hostPort 80 protocol tcp appProtocol http essential true command prefect worker start pool test pool environment name PREFECT API KEY value name PREFECT API URL value name EXTRA PIP PACKAGES value prefect aws s3fs environmentFiles mountPoints volumesFrom ulimits logConfiguration logDriver awslogs options awslogs create group true awslogs group ecs agent task awslogs region eu central 1 awslogs stream prefix ecs family agent task executionRoleArn arn aws iam 759317626178 role ecsTaskExecutionRole networkMode awsvpc revision 8 volumes status ACTIVE requiresAttributes name com amazonaws ecs capability logging driver awslogs name ecs capability execution role awslogs name com amazonaws ecs capability docker remote api 1 19 name com amazonaws ecs capability docker remote api 1 18 name ecs capability task eni name com amazonaws ecs capability docker remote api 1 29 placementConstraints compatibilities EC2 FARGATE requiresCompatibilities FARGATE cpu 1024 memory 3072 runtimePlatform cpuArchitecture X86 64 operatingSystemFamily LINUX registeredAt 2023 04 26T17 33 57 713Z registeredBy arn aws iam 759317626178 root tags ```

Zanie · Answer

I don t think ECS respects `EXTRA PIP PACKAGES` because it bypasses our entrypoint

Yaron Levi · Answer

Let me understand because I am little bit confused

Yaron Levi · Answer

I ve created a new work pool like so

Yaron Levi · Answer

CleanShot 2023 04 26 at 21 41 05 2x jpg

Zanie · Answer

I think if you add ` opt prefect entrypoint sh` at the start of your commands it ll install `prefect aws` like you want

Yaron Levi · Answer

It runs nice on ECS and listens to new Runs

Zanie · Answer

I see I don t think you can set an infrastructure block when using non agent work pool types cc < alex>

Yaron Levi · Answer

How should I define a deployment so that it will be picked up by this worker pool

Yaron Levi · Answer

Ah so it s not possible to create a new ECS Task for each Run

Yaron Levi · Answer

Meaning when the work pool task polled for a new flow it should create a new ECS Task

Zanie · Answer

We do create a new ECS task for each run

Zanie · Answer

That s how the ECS worker functions

Yaron Levi · Answer

Ok so what I am defining wrong here

Zanie · Answer

Workers don t use infrastructure blocks in the same way that agents do

Zanie · Answer

The work pool defines the infrastructure instead

Yaron Levi · Answer

ok so instead of ```prefect deployment build jobs test1 test1 py test1 name test1 output jobs test1 deployment storage block github yuvital prefect github pool test pool infra block ecs task ecs test block ``` I should do ```prefect deployment build jobs test1 test1 py test1 name test1 output jobs test1 deployment storage block github yuvital prefect github pool test pool```

Yaron Levi · Answer

removing the infra from the deployment

alex · Answer

To create a deployment in an `ecs` work pool you ll want to use the new Prefect Projects functionality Following the project tutorial is a good way to get started <https docs prefect io latest tutorials projects >

Yaron Levi · Answer

ok got it and following this method when the worker pool get a new Run it will create a new ECS Task for it

alex · Answer

Yes the ECS worker will create a new ECS for each run Looking at your original error I m surprised you got that error for this deployment because workers should not execute flow runs from deployments with a storage block Do you have any other agents polling this work pool by any chance

Yaron Levi · Answer

No I ve checked Nobody else is polling this work pool

Yaron Levi · Answer

And also looking at the logs of the work pool

Yaron Levi · Answer

Like you ve said

alex · Answer

Gotcha that s a bug It shouldn t have executed that flow run Would you please submit an issue for this

Yaron Levi · Answer

Sure I ll try I am still trying to wrap my head around all the moving parts