https://prefect.io logo
Join Slack
Powered by
Where does the Docker storage inherit its credenti...
# ask-community
j
Where does the Docker storage inherit its credentials from? I assume boto3, but I'm getting a weird behavior on Github Actions. Using the manual docker build and push with the ECR Login action (https://github.com/aws-actions/amazon-ecr-login#usage), I can push my image just fine to ECR. However, when I run this with flow.register() it fails on no authorization.
Copy code
raise InterruptedError(line.get("error"))
      InterruptedError: no basic auth credentials
Well, actually then the error would also occur locally but it works. OK, now I'm stumped again
k
Are you doing the
Copy code
aws ecr get-login-password
?
j
Yup
Even tried with a manual login instead of the Github Action, inlining in the prefect register step in GA. Still failed. Odd
k
This login only lasts 12 hours right?
j
Sure, but each run it gets renewed.
Since each run triggers the ecr-login in gha
k
Let me answer the first question though. Docker storage basically will work if 
docker push
works
j
That's not what I'm seeing. I just tagged and pushed via docker push. And using the same environment, registering the flow fails with no basic auth
k
That’s pretty weird. You can find the code here, it should but be making a Docker client and pushing. That Docker Client should just be using Docker in the same environment. Maybe you can try using the Python 
client.push()
and if it fails, we can look into why 
docker-py
isn’t getting the credentials
j
Copy code
AWS_PROFILE=github make docker-login
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <http://EDITED.dkr.ecr.us-east-1.amazonaws.com|EDITED.dkr.ecr.us-east-1.amazonaws.com>
WARNING! Your password will be stored unencrypted in /home/EDITED/.docker/config.json.
Configure a credential helper to remove this warning. See
<https://docs.docker.com/engine/reference/commandline/login/#credentials-store>
Login Succeeded
Copy code
Successfully tagged <http://EDITED.DKR.ECR.US-EAST-1.AMAZONAWS.COM/EDITED-prod-platform-prefect:latest|EDITED.DKR.ECR.US-EAST-1.AMAZONAWS.COM/EDITED-prod-platform-prefect:latest>
[2022-04-22 13:03:59-0500] INFO - prefect.Docker | Pushing image to the registry...
    Error building storage:
      Traceback (most recent call last):
        File "/home/EDITED/.local/share/virtualenvs/data-vrc-T2jvyYmD/lib/python3.9/site-packages/prefect/cli/build_register.py", line 457, in build_and_register
    storage.build()
        File "/home/EDITED/.local/share/virtualenvs/data-vrc-T2jvyYmD/lib/python3.9/site-packages/prefect/storage/docker.py", line 325, in build
    self._build_image(push=push)
        File "/home/EDITED/.local/share/virtualenvs/data-vrc-T2jvyYmD/lib/python3.9/site-packages/prefect/storage/docker.py", line 399, in _build_image
    self.push_image(full_name, self.image_tag)
        File "/home/EDITED/.local/share/virtualenvs/data-vrc-T2jvyYmD/lib/python3.9/site-packages/prefect/storage/docker.py", line 613, in push_image
    raise InterruptedError(line.get("error"))
      InterruptedError: no basic auth credentials

  Registering 'hello-flow'... Error
================== 0 registered, 1 errored ==================
k
I can’t really find anything about why 
docker-py
isn’t picking it up. This is a widely used setup among Prefect users though so this is very weird
j
Yeah - tried a few more things but ended up replicating. I can docker push fine in my terminal after ecr-login, but I can't do a prefect register --project .. --path '*. to register the flow
@Kevin Kho: i think it's related to the Linux credential store. The flow registers fine on MacOS, which uses the MacOS keychain. But on Github actions (ubuntu) and my desktop (ubuntu), docker-py gives the same error.
k
Man not sure I can advise on that. Do you have multiple log-ins or something? I feel like I have seen people deploy with Github actions though
a
it's worth opening an issue, I see this error the second time today @Marvin open "Flow registration with Docker storage can't push the image to ECR due to `InterruptedError: no basic auth credentials`" @Jason Motley as a workaround, for now, you could use the solution from the Discourse topic I linked, i.e. manually pushing the image yourself to ECR
m
j
There hasn't been a release since 2021, so I don't think the issue is docker-py, per se then: https://github.com/docker/docker-py/releases/tag/5.0.3
Is there anybody that does have a GA ECR push that would share their workflow, alternatively? Perhaps I'm setting it up incorrectly.
k
Let’s look for one around here
a
you said it works when you push manually, but doesn't work when you do that with flow.register()? if not, please clarify your issue a bit more Jason
👍 1
k
But not ECR. Will continue looking
j
That's correct. I suppose I can just add a step in GHA to tag and push the image, and have .register() use the same tag and it should just pull it
k
That’s true
85 Views
Open in Slack
PreviousNext
Powered by