https://prefect.io logo
Title
j

John T

08/06/2021, 5:50 AM
Hello! I’m having a hard time with containerizing my prefect docker agent. I will post more information in the thread
n

nicholas

08/06/2021, 6:00 AM
Hi @John T - can you describe what issue you're having? It also might be easier to use the
prefect agent start docker --show-flow-logs
command as your entrypoint instead of calling your script.
j

John T

08/06/2021, 6:09 AM
Hello @nicholas thanks for getting back so quickly. I will paste that right now
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
but I will paste the full traceback
hope the traceback helps
I add the
--show-flow-logs
but no luck
prefect agent docker start --show-flow-logs
works without an issue when running it straight from my terminal, but when I have that as the
entrypoint
, it fails and I get said traceback
k

Kevin Kho

08/06/2021, 1:34 PM
What Prefect version are you on in the container where you start the agent?
j

John T

08/06/2021, 2:38 PM
Hi @Kevin Kho, its 0.15.3
z

Zanie

08/06/2021, 3:11 PM
Hi, the
No such file or directory
error is indicating that the Docker API socket is not available
This is because you're running docker-in-docker
j

John T

08/06/2021, 5:09 PM
Michael thanks! So how would I go about containerizing my agent? I’m sure I’m thinking of this wrong, but if I can’t containerize this then how would I run this in a remote machine
z

Zanie

08/06/2021, 5:17 PM
You can containerize the agent but you'll need to expose the docker interface from the host os to the container; I think the
--privileged
when starting the agent container should accomplish it. See https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
j

John T

08/06/2021, 5:50 PM
Awesome, thanks. I will look into this
And just to confirm this wouldn’t be an issue if I was containerizing the ECSAgent?
As a note; running with
--privileged
still gives me the same issue
z

Zanie

08/06/2021, 6:10 PM
Yeah containerizing a non docker agent should be fine yeah
If you exec into the container you can start the agent without issue? Can you run docker commands?
j

John T

08/06/2021, 7:10 PM
i get
docker not found
error
I’m guessing I would have to install docker?
z

Zanie

08/06/2021, 7:24 PM
Yeah you'd need the docker CLI to test that. To be clear, you shouldn't need it installed to containerize the agent though, dockerpy is included and is trying to connect to the docker-engine socket from your host system.
k

Kieran

09/01/2021, 6:50 PM
This looks like the same issue I am running into, re: running docker in docker. Has there been an underlying change to how dockerpy works?
k

Kevin Kho

09/01/2021, 6:53 PM
I don’t think there has been a change. This has always been a tricky situation (docker in docker). This user mentioned they got past it but didn’t give much details.
k

Kieran

09/01/2021, 8:01 PM
Thanks @Kevin Kho for sharing that. I am not too sure on how to mount the docker.sock... I do currently use that same aws + docker script to login. But the container running the healthcheck is not able to connect to the docker daemon to verify base_url and tls config. Or, so that is what I understand my problem to be...
k

Kevin Kho

09/01/2021, 8:03 PM
Yeah I haven’t tried it myself but maybe you agent on the agent like this ?
I see you pinged him. I’d like to learn myself 😆
k

Kieran

09/01/2021, 8:05 PM
Ha 🙏 Would I look to add
docker run -it -v /var/run/docker.sock:/var/run/docker.sock ubuntu:latest sh -c "apt-get update ; apt-get install <http://docker.io|docker.io> -y ; bash"
as a
extra_dockerfile_commands
in the Docker class? to run prior to the healthcheck?
k

Kevin Kho

09/01/2021, 8:06 PM
If this
-v /var/run/docker.sock:/var/run/docker.sock
is the important part, I think it would be mounted on the agent here ?
Notice the end of that page also with the Docker Daemon Address. It could be that….but he did say mount so I think it’s through
-v
k

Kieran

09/01/2021, 9:55 PM
I think I'm solving the same but slightly different problem. My docker in a docker scenario is generated by using docker storage and circleCi to register my images. I think... I need to mount my images using the extra docker commands to the daemon port socket. So that when the healthcheck runs it is able to pull the base url and tls from docker. My eyes are going a bit squire trying to fix this so I'll have a test in the morning.
k

Kevin Kho

09/01/2021, 9:59 PM
Ah ok I see
k

Kieran

09/01/2021, 10:03 PM
This thread @Kevin Kho
k

Kevin Kho

09/01/2021, 10:04 PM
Did you mean another thread?
k

Kevin Kho

09/01/2021, 10:41 PM
Ah ok. I didn’t read through that a while ago. What is your RunConfiguration? and the is the image of that? I think using a base image like
prefecthq/prefect:python.3.6
would pull that 3.6.14
k

Kieran

09/02/2021, 7:03 AM
For the run config we use
ECSRun()
and pull images from an ECR repo. I have just noticed that the CI logs do show:
FROM prefecthq/prefect:0.15.4-python3.6
when building the image. How do I control that @Kevin Kho?
k

Kevin Kho

09/02/2021, 2:07 PM
We don’t have versions for every Python version so you would likely need your own image. Maybe you can try making a
pipenv
and activating it? Conda is hard to activate, pipenv might be easier but i’m not sure.