https://prefect.io logo
Join Slack
Powered by
Okay, it seems there is a gap in my Prefect unders...
# ask-community
e
Okay, it seems there is a gap in my Prefect understanding. I've got a docker image with a flow inside that works locally. I'd like to run it with the ECSAgent. I have the ECS agent set up in AWS. What is the process for actually deploying my flow now?
k
Hey @Eric Mauser, I think it will be 1. Register the flow with a Storage defined. This can be Docker/S3/Github. Some place that the ECS agent can pull it from. 2. Switch to the 
ECSRun
Run Configuration. This takes in an image . Just make sure that it can pull the image. 3. Make sure you have a role with the appropriate permissions. You can pass this to the 
ECSRun
also. 4. Queue a Flow Run and it should be picked up by the ECS agent (just make sure labels are the same), and then start the ECS task.
This is a good read if you haven’t seen it yet.
z
the one gotcha for me was the label on your agent must match the label on the flow youre registering. E.g. if your agents label is 
ecs-dev
you must also register your flow with the label 
ecs-dev
👍 1
e
thanks for the quick responses! I think I was overcomplicating it and combining the flow steps with the docker code. I'll work through these today and ping back if I find more issues
Alright, almost cleared up. Getting this error when registering the flow. Looks like it is happening at the pushing of the docker image. I'm assuming it's the credentials for the ECR repo. Does the agent need these credentials? or where do these need to go?
Copy code
[2021-07-26 08:12:48-0700] INFO - prefect.Docker | Pushing image to the registry...
Traceback (most recent call last):
  File "flow.py", line 18, in <module>
    flow.register('eqtble_test')
  File "/Users/emauser/.pyenv/versions/prefect_ecs_venv/lib/python3.7/site-packages/prefect/core/flow.py", line 1734, in register
    idempotency_key=idempotency_key,
  File "/Users/emauser/.pyenv/versions/prefect_ecs_venv/lib/python3.7/site-packages/prefect/client/client.py", line 1105, in register
    serialized_flow = flow.serialize(build=build)  # type: Any
  File "/Users/emauser/.pyenv/versions/prefect_ecs_venv/lib/python3.7/site-packages/prefect/core/flow.py", line 1497, in serialize
    storage = self.storage.build()  # type: Optional[Storage]
  File "/Users/emauser/.pyenv/versions/prefect_ecs_venv/lib/python3.7/site-packages/prefect/storage/docker.py", line 303, in build
    self._build_image(push=push)
  File "/Users/emauser/.pyenv/versions/prefect_ecs_venv/lib/python3.7/site-packages/prefect/storage/docker.py", line 377, in _build_image
    self.push_image(full_name, self.image_tag)
  File "/Users/emauser/.pyenv/versions/prefect_ecs_venv/lib/python3.7/site-packages/prefect/storage/docker.py", line 585, in push_image
    raise InterruptedError(line.get("error"))
InterruptedError: no basic auth credentials
k
You need to do something like this:
Copy code
aws ecr get-login-password --region <REGION> | docker login -- username AWS --password-stdin <ACCOUNT>.dkr.ecr.<REGION>.<http://amazonaws.com|amazonaws.com>
to get the log-in to push.
e
thanks Kevin, that worked for me (minus the extra space before username). Here is the corrected command for future readers
Copy code
aws ecr get-login-password --region <REGION> | docker login --username AWS --password-stdin <ACCOUNT>.dkr.ecr.<REGION>.<http://amazonaws.com|amazonaws.com>
👍 1
k
Just note that log-in is only good for 12 hours
I think you need to use this for production authentication
Open in Slack
PreviousNext
Powered by