https://prefect.io logo
Join Slack
Powered by
Hi Prefect Community, I’m using Docker storage and...
# ask-community
f
Hi Prefect Community, I’m using Docker storage and I’m running the Prefect ECS Agent on an AWS Fargate cluster successfully. Whenever I register a flow I need to authenticate my docker client. At the moment I’m doing that by using the AWS CLI and running 
aws ecr get-login-password --region regionname | docker login --username AWS --password-stdin <http://xxxxxx.dkr.ecr.us-east-2.amazonaws.com|xxxxxx.dkr.ecr.us-east-2.amazonaws.com>
. I’ve tried using boto3 to do a docker login right before the 
register()
call but it seems like the auth token expires as soon as I run register:
Copy code
denied: Your authorization token has expired. Reauthenticate and try again.
Is it possible to have Prefect do the docker login? Or do I always have to use the AWS CLI?
z
Hi @Fina Silva-Santisteban -- How long is the token expiring after?
f
@Zanie I’m not sure about the exact duration, it seems like seconds? This is how it currently looks like
Copy code
def connect_docker_client_to_cloud():
    (some boto3 setup)
    (...)
    login_result = dockerClient.login(username=username, password=password, registry=registry)
    print(login_result)  # Returns {'IdentityToken': '', 'Status': 'Login Succeeded'}


def register_flows
	(some prefect setup)
	(...)
	connect_docker_client_to_cloud()
	flow.register(project_name=project_name) #Returns denied: Your authorization token has expired.
z
Alright, there must be something wrong with the token then. per https://docs.aws.amazon.com/cli/latest/reference/ecr/get-login-password.html the token should be valid for 12 hours
Just to confirm, running it via shell is successful but via a python function that calls 
dockerClient.login
is not?
f
@Zanie interesting! Yes it works fine via shell! The strange thing is that the python function returns 
login succeeded
as well. 🤔
z
Perhaps you need to pass
Copy code
reauth (bool): Whether or not to refresh existing authentication on
    the Docker server.
I'm a little uncertain what's actually going on behind the scenes with docker's authentication
It seems possible that you're logging in instance of the docker client that you have there but when the docker storage is built during 
flow.register
it is creating a new instance of the docker client that does not have your authentication attached
Here's a snippet from the 
Docker(Storage)
class
Copy code
def _get_client(self) -> "docker.APIClient":
        # 'import docker' is expensive time-wise, we should do this just-in-time to keep
        # the 'import prefect' time low
        import docker

        client = docker.APIClient(
            base_url=self.base_url, version="auto", tls=self.tls_config
        )
        client.login()
You could try subclassing 
Docker
storage and overriding this function to call 
login
with your args -- I'm not sure that'll work with serialization and such though. I'll send a message to the rest of the team asking about this pattern.
f
@Zanie That sounds reasonable! So there must be a difference then between using boto3 and aws cli when doing the docker login. (?) Thank you for asking the rest of your team!
z
You could also try running 
docker login ...
in a subprocess/shell from python. I believe it's just writing auth information to a file which is then pulled if you call 
client.login()
without args.
f
@Zanie setting 
reauth
didn’t change anything either! I’ve also tried a few different to do the docker login according to what I found on stackoverflow, but no luck either. I guess I’ll have to go with the aws cli docker login for now. 😓 In case anybody else stumbles upon this, the command is this one:
Copy code
aws ecr get-login-password --region yourregion | docker login --username AWS --password-stdin <http://yourregistryid.dkr.ecr.yourregion.amazonaws.com|yourregistryid.dkr.ecr.yourregion.amazonaws.com>
3 Views
Open in Slack
PreviousNext
Powered by