Hello Eveveryone! I am using a CI/CD process with AWS Codepipeline (source: Github, build: Codebuild) the output is a docker image i push to an AWS ECR repository for each flow. I my github project i have a buildspec file where i put this code:

build:
    commands:
      - python --version
      - pyenv global 3.7.7
      - python --version
      - pip install $(grep -ivE "pywin32" requirements.txt)
      - prefect backend cloud
      - prefect auth login -t $PREFECT_ACCESS_TOKEN
      - aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin <http://xxxxxxxx.dkr.ecr.us-east-2.amazonaws.com|xxxxxxxx.dkr.ecr.us-east-2.amazonaws.com>
      - python3.7 flows/my_flow.py

From November 20 docker introduced pull limits for images what cause me to have this error in Code build

File "/root/.pyenv/versions/3.7.7/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 619, in pull_image
    output = client.pull(self.base_image, stream=True, decode=True)
  File "/root/.pyenv/versions/3.7.7/lib/python3.7/site-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/root/.pyenv/versions/3.7.7/lib/python3.7/site-packages/docker/api/client.py", line 261, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/root/.pyenv/versions/3.7.7/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error: Internal Server Error ("toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: <https://www.docker.com/increase-rate-limit>")

When i go to the site i can read: "Anonymous free users will be limited to 100 pulls per six hours, and authenticated free users will be limited to 200 pulls per six hours." So i created a account on docker to raise my limits to 200 pull per 6 hours. So i understand the problem occurs when prefect is pulling the base prefect docker image to build my flow image because i have reached my 100 pulls / 6 hours anonymous limit My question is where should i put my docker account token to pull the prefect base image via my newly created docker account?

Hi! We’re just using the

docker-py

API package. It provides a

login

method, but we haven’t set anything up to use that. I’m curious if you use the

docker login

command before building your flow if it’ll store and use the credentials.

To clarify Michael's 2nd suggestion, you can copy the

prefecthq/prefect

base image into ECR, then pass the registry & name for the ECR version as the

base_image

kwarg to the

Docker

storage. This would let you avoid ever hitting dockerhub.

Wonderful!. I just tested the 2nd suggestion. It works flawlessly. Thanks a lot @Zanie and @Jim Crist-Harif.