Has anyone had any luck creating a `DockerRegistry` block wi

Question

Has anyone had any luck creating a `DockerRegistry` block with AWS ECR

Dev Dabke · Answer

I am also working on this

Joshua Grant · Answer

I was following this issue on github which brought be to this PR but was wondering if there was a workaround for now

Anna Geller · Answer

it s on our radar afaik < alex> team has it on their backlog until then I recommend using ECSTask with ECR being part of it natively we have a repo + blog post + video here

Nikhil Jain · Answer

< Anna Geller> what storage type block should I use with ECSTask when my flow code is in a ECR image

Anna Geller · Answer

hahaha

Anna Geller · Answer

are you trying to see if me or Marvin gives a better answer joy

Nikhil Jain · Answer

I don t think Marvin can beat you anytime soon slightly smiling face I just thought you might be on vacation since you were not online so I tried asking marvin as well slightly smiling face

Anna Geller · Answer

the right and easiest solution to this would likely be to point the path to the path on the image so e g on your deployment e g if on the image myflow py is baked into this location opt prefect flows myflow py then this should work ```prefect deployment build n dev a ib ecs task dev myflow py myflow path opt prefect flows skip upload```

Nikhil Jain · Answer

Thanks couple of followup questions 1 if I supply a `task definition arn` to my ECSTask block which all of the settings in the task definition be applied automatically or do I have to supply some of the settings in the ECSTask block settings 2 e g where should I add the ECR image url in the ECSTask block or in the `task definition` which I am creating using terraform

Anna Geller · Answer

Good question and one for which Andrew has recently added a PR to clarify <https github com PrefectHQ prefect aws commit cdd40bdacec5465050e3fc4f174f4301db294ebc> if you register a task definition yourself you set the image on the task definition if you let Prefect create the task definition for you you specify the image on the ECSTask block

Nikhil Jain · Answer

How does the agent know what `command` to run in the ECS Task Do I have to hardcode the command in the ECR image or in the task definition Or will the agent automatically add a `command` override when creating the task

Anna Geller · Answer

another good question and I stumbled upon it myself lately No need to do that Even if your image would have say entrypoint like ```ENTRYPOINT prefect agent start q default ``` any Infrastructure block by default overrides it to ``` python m prefect engine ```

Nikhil Jain · Answer

I made some progress however now getting an unexpected error in the `agent` logs ``` File usr local lib python3 10 site packages prefect aws ecs py line 693 in create task and wait for start network config = self load vpc network config self vpc id boto session File usr local lib python3 10 site packages prefect aws ecs py line 1228 in load vpc network config vpcs = ec2 client describe vpcs describe Vpcs File usr local lib python3 10 site packages botocore client py line 508 in api call return self make api call operation name kwargs File usr local lib python3 10 site packages botocore client py line 915 in make api call raise error class parsed response operation name botocore exceptions ClientError An error occurred UnauthorizedOperation when calling the DescribeVpcs operation You are not authorized to perform this operation ``` My flow is setup to run on FARGATE I noticed that this stack trace is using `ec2 client` Is this expected The `task execution role` for my agent does not have any `ec2 ` capabilities and used to work fine on Prefect1 with the same network config

Anna Geller · Answer

I d need to see the full block config

Anna Geller · Answer

generally this is needed to start a task because even if you have a task definition you need to run it in a specific subnet and to do that we check VPC network config unless you specify the subnets explicitly I think

Nikhil Jain · Answer

ah that s very helpful I need to add the subnets and vpc id as task customizations

Nikhil Jain · Answer

I was able to get my flows working on ECS Couple of additional notes for anyone trying to do this 1 The name of the ECS task container in which the flow is supposed to be run should be prefect 2 I added vpc id in the ECSTask params and aded subnet ids and security group ids in `networkConfiguration` as task customizations 3 turns out <https github com PrefectHQ prefect aws blob main prefect aws ecs py L1226|ECSTask calls> `ec2 client describe vpcs` even when vpc id is provided explicitly So had to add that policy to the task execution role

Anna Geller · Answer

Thanks sooo much and nice work Interesting find about describe vpc we ll need to cross check

Anna Geller · Answer

< Nikhil Jain> I answered this here in more detail