Bogdan Bliznyuk
02/18/2022, 11:50 AMaz acr login
But, it seems that prefect docker agent only reads the token during start and stores it in memory. Unless we restart the prefect docker agent, it is unable to pull docker image flows after 3hrs (the acr token's expired)Anna Geller
docker pull yourCustomACRimage
Bogdan Bliznyuk
02/18/2022, 12:10 PMdocker pull
works
it seems that prefect's using the python DockerApi client and it doesn't refresh the in-memory credentialsAnna Geller
Bogdan Bliznyuk
02/18/2022, 12:14 PMdocker pull
without any authentication on each VM
but prefect agent doesn't pick up the refreshed tokenAnna Geller
Bogdan Bliznyuk
02/18/2022, 12:36 PMMarvin
02/18/2022, 12:36 PMBogdan Bliznyuk
02/18/2022, 12:38 PMAnna Geller
Bogdan Bliznyuk
02/18/2022, 12:41 PMaz acr login
, so you're able to do docker pull
at any given time
but prefect docker agent only reads the token when it starts. after 3 hours this tokens because it's expiredAnna Geller
Bogdan Bliznyuk
02/18/2022, 12:43 PMdocker login
or az acr login
before pulling or pushing to/from the ACRAnna Geller
Bogdan Bliznyuk
02/18/2022, 12:47 PMAnna Geller
Bogdan Bliznyuk
03/03/2022, 1:58 PMAnna Geller
az acr login
, but this guide shows how you can generate long-lived credentials that will persist and the way to do it is to use docker login with a service principal credentials:
docker login <http://prefectcommunity.azurecr.io|prefectcommunity.azurecr.io> -u $USER_NAME -p $PASSWORD
I can 100% guarantee that if you follow this on a new VM, you won’t have to restart your agent every 3 hours and you will not have to login again to ACR every 3 hours. In fact, I used the same approach on my local machine and I haven’t logged in for 3 days and I can still push new images to ACR.
The likely issue that you may have is related to Azure CLI version that I covered in this section: https://discourse.prefect.io/t/how-to-spin-up-a-docker-agent-on-azure-vm-a-full-walkthrough/407#troubleshooting-tips-8
It only works if you use Azure CLI version 2.25.0 or later
Azure unfortunately doesn’t make this process easy and they don’t document it well enough in my opinion, but I’m 100% positive that this approach generates long-lived permissions that persist. I had this agent running for 2 days and I didn’t have to login a single time after setting it up the first time.
And again, I really understand your frustrations because I went through all the same pain as you did 😄Bogdan Bliznyuk
03/03/2022, 2:14 PMdocker login
instead az acr login
.
this actually a good point and should work. we'll try it out and let you know.
thank you very much!!!