https://prefect.io logo
Powered by
Do I need a custom ECR image for my agent or can my ECS task running agent be based on the base Prefect image?
k

Kelvin DeCosta

10/20/2022, 4:42 PM
Hey guys! I have a Prefect Agent running as an ECS Service. It is uses a lightweight container and will be invoking deployments via 
ECSTask
. My question: Do I need to install dependencies like 
prefect-aws
, 
s3fs
etc. and register the blocks (eg: 
prefect block register -m prefect_aws.ecs
) in the container for the agent? Or can I just use the latest 
prefecthq/prefect
image?
👍 1
✅ 1
I'm currently using this Dockerfile: 
FROM prefecthq/prefect:2-python3.10

# Install dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt --no-cache-dir

# Register AWS block types
RUN prefect block register -m prefect_aws.ecs
But I think this might be unused / redundant
a

Anna Geller

10/20/2022, 4:43 PM
Your intuition is right, you can use the latest 
prefecthq/prefect
image for your agent
✅ 1
the ECSTask block has all attributes needed, agent can be super lightweight and only needs prefect as a dependency we have this repo template with a blog post and video tutorial linked in the readme https://github.com/anna-geller/dataflow-ops
k

Kelvin DeCosta

10/20/2022, 4:47 PM
Thank you! And yes, this repo has been instrumental to our approach! I just ran my first prefect deployment as an ECS Task yesterday! Now, I'm setting up some workflows to keep things updated
:thank-you: 2
👍 3
a

Anna Geller

10/20/2022, 4:52 PM
awesome, so great to hear and keep us posted how it goes! 🙌
k

Kelvin DeCosta

10/25/2022, 5:26 PM
Hey Just wanted to update that you do infact need to install 
prefect-aws
if you plan to use 
ECSTask
. I ran into a 
KeyError: ecs-task
could not be dispatched Also, I recently switched from 
S3
to private 
GitHub
and I had to install 
git
on the 
prefect
container's image in my 
ECSTask
definition
7 Views
#prefect-communityJoin Slack
Powered by