Joshua Grant
11/18/2022, 5:30 PMdataflow-ops
repo and have tried to use the ECSTask
from prefect-aws
to circumvent the issue surrounding DockerRegistry
for ECR, but I keep getting the error KeyError: "No class found for dispatch key 'ecs-task' in registry for type 'Block'"
. Prefect==2.6.8 & prefect-aws==0.1.8ecs-task
is the name of my ECSTask
Ryan Peden
11/18/2022, 5:38 PMprefect-aws
package installed on the machine where your agent is running?Joshua Grant
11/18/2022, 5:40 PMRyan Peden
11/18/2022, 5:41 PMJoshua Grant
11/18/2022, 5:43 PMFROM prefecthq/prefect:2-python3.9 # example base image
RUN pip install s3fs prefect-aws
RUN prefect block register -m prefect_aws.ecs
or just the agent?Ryan Peden
11/18/2022, 6:00 PMFROM prefecthq/prefect:2-python3.9 # example base image
RUN pip install s3fs prefect-aws
The prefect-aws
collection should auto-register ECSTask with the local install of Prefect, so I don't think you'll to do it again. The prefect block register -m prefect_aws.ecs
command registers it with Prefect cloud or Orion server but it sounds like you've already done that.
It's probably easiest to use the same image everywhere, but if you are using the default S3
storage block I believe that for flow runs you would only need:
FROM prefecthq/prefect:2-python3.9 # example base image
RUN pip install s3fs
If you go that route, double check which block you are using for flow storage; prefect-aws
contains a storage block named S3Bucket
that has a few more features than the S3
block bundled with Prefect - but you'd know if you were using S3Bucket
because you'd need to register prefect_aws.s3
first.
(Sorry for the long reply; just want to make sure I cover everything that might be useful)