Hey all! Having issues getting the DockerAgent att...
# ask-communitys
Samuel Hinton
02/10/2021, 1:15 PMHey all! Having issues getting the DockerAgent attempting to do some DockerRun flows not being able to access the registry. We have a set of images in an Amazon ECR location, but I cant see any registry setting options in the DockerRun or DockerAgent (and having the
AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYj
Jan Marais
02/10/2021, 1:43 PMIs your Docker client authenticated with ECR?
Jan Marais
02/10/2021, 1:44 PMFor e.g. with:
Copy code
aws ecr get-login-password --region region | docker login --username AWS --password-stdin <http://aws_account_id.dkr.ecr.region.amazonaws.com|aws_account_id.dkr.ecr.region.amazonaws.com>s
Samuel Hinton
02/10/2021, 1:45 PMI assume that is indeed the issue - the docker agent doesnt have an authentication step that I can see, and the tokens you get from that last 12 hours only, so it cant be something I run in the command manually before launching
prefect agentj
Jan Marais
02/10/2021, 1:51 PMSorry, I only use docker agents for debugging and never had the need for leaving them running > 12 hours. Maybe the information here will lead you to a solution: https://docs.aws.amazon.com/AmazonECR/latest/userguide/registry_auth.html
Jan Marais
02/10/2021, 1:52 PMNote that simply providing an access key id and secret will not be enough to authenticate pull and push with docker and ECR
s
Samuel Hinton
02/10/2021, 1:53 PMOh, does it need something else?
j
Jan Marais
02/10/2021, 1:54 PMSee second paragraph of page following link above
Jan Marais
02/10/2021, 1:54 PM"The Docker CLI doesn't support native IAM authentication methods. Additional steps must be taken so that Amazon ECR can authenticate and authorize Docker push and pull requests."
s
Samuel Hinton
02/10/2021, 1:55 PMAh right, gotcha. Will hopefully see if someone else has managed to get this working within a DockerAgent
k
Karolína Bzdušek
02/10/2021, 2:01 PMThis is just a thought, not sure whether this would help (right now I am trying to understand where and how to authenticate everything that it would run smoothly), but except setting env variables, have you share this in secrets in UI? I have my GITHUB_ACCES_TOKEN exported in env variables but it seems that having share it as a secret in UI actually helps..It's just a wild guess, anyone who can enlighten me regarding authentication is very welcome.
s
Samuel Hinton
02/10/2021, 2:02 PMI believe the issue is that the AWS authentication is something external to docker, and thus harder to integrate. The AWS cli needs to actually be called and passed to dockers auth, and then for 12 hours Id get to utilise the ECR
j
Jan Marais
02/10/2021, 2:09 PMMaybe this will help: https://github.com/awslabs/amazon-ecr-credential-helper
Seems like it will automatically fetch the credentials for ECR using your standard IAM credentials
🙏 1
s
Samuel Hinton
02/10/2021, 2:11 PMIll see if I can get this working
Samuel Hinton
02/10/2021, 2:11 PMCheers!
Samuel Hinton
02/10/2021, 4:02 PM@Jan Marais just wanted to chime back in and say this seems to help, and I can see the docker agent “Successfully pulled image 59630237498……“. It still doesnt run the flow, but at least it now pulls the image down and starts the container!
j
Jan Marais
02/10/2021, 6:00 PMGlad you are making progress. Do you get any indication why the flow is not running?
s
Samuel Hinton
02/10/2021, 7:09 PMYes I believe there is a networking issue because the docker-compose being run is actually part of a larger swarm which changes the network name and makes it unattachable. Ill slap it into shape soon Im sure