https://prefect.io logo
Title
j

Jonathan Mathews

04/05/2022, 3:50 PM
Hi, I’m following the instructions in this great article on using ECS. I’m now setting up a custom docker image to run dbt (I’m assuming I can’t use
DbtShellTask
because dbt wouldn’t have been installed). It’s coming back with an error where I need to specify a
execution_role_arn
in the
ECSRun
which I’ve now done, but can I use the previously defined role, or do I need to define something else? Thanks!
k

Kevin Kho

04/05/2022, 3:55 PM
You can just add the dbt library to the image. If you don’t want to make your own image, you can just use the base prefect image and add the env var
EXTRA_PIP_PACKAGES
which takes in packages to install. You can find that here , only works if you use the base prefect image. But could you show me the ECSRun code? Just remove sensitive info
a

Anna Geller

04/05/2022, 6:11 PM
this post seems to answer your question
the only difference is that this post is using EKS rather than ECS Fargate. If you use EKS on Fargate, you can follow the above article 1:1 with the only difference that the cluster could be created using the Fargate profile with
--fargate
flag:
eksctl create cluster --fargate
j

Jonathan Mathews

04/05/2022, 6:47 PM
Fantastic! Thanks both. I actually managed to get it working using the initial post. I’d just skipped a step somehow. I also didn’t know about the
EXTRA_PIP_PACKAGES
env var, so that’s super helpful to know. Thanks
👍 1