Hello! I’m having a hard time with containerizing my prefect docker agent. I will post more information in the thread

Hi @John T - can you describe what issue you're having? It also might be easier to use the

prefect agent start docker --show-flow-logs

command as your entrypoint instead of calling your script.

Hello @nicholas thanks for getting back so quickly. I will paste that right now

What Prefect version are you on in the container where you start the agent?

Hi @Kevin Kho, its 0.15.3

Hi, the

No such file or directory

error is indicating that the Docker API socket is not available

Michael thanks! So how would I go about containerizing my agent? I’m sure I’m thinking of this wrong, but if I can’t containerize this then how would I run this in a remote machine

You can containerize the agent but you'll need to expose the docker interface from the host os to the container; I think the

--privileged

when starting the agent container should accomplish it. See https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities

Awesome, thanks. I will look into this

Yeah containerizing a non docker agent should be fine yeah

i get

docker not found

error

Yeah you'd need the docker CLI to test that. To be clear, you shouldn't need it installed to containerize the agent though, dockerpy is included and is trying to connect to the docker-engine socket from your host system.

This looks like the same issue I am running into, re: running docker in docker. Has there been an underlying change to how dockerpy works?

I don’t think there has been a change. This has always been a tricky situation (docker in docker). This user mentioned they got past it but didn’t give much details.

Thanks @Kevin Kho for sharing that. I am not too sure on how to mount the docker.sock... I do currently use that same aws + docker script to login. But the container running the healthcheck is not able to connect to the docker daemon to verify base_url and tls config. Or, so that is what I understand my problem to be...

Yeah I haven’t tried it myself but maybe you agent on the agent like this ?

Ha 🙏 Would I look to add

docker run -it -v /var/run/docker.sock:/var/run/docker.sock ubuntu:latest sh -c "apt-get update ; apt-get install <http://docker.io|docker.io> -y ; bash"

as a

extra_dockerfile_commands

in the Docker class? to run prior to the healthcheck?

If this

-v /var/run/docker.sock:/var/run/docker.sock

is the important part, I think it would be mounted on the agent here ?

I think I'm solving the same but slightly different problem. My docker in a docker scenario is generated by using docker storage and circleCi to register my images. I think... I need to mount my images using the extra docker commands to the daemon port socket. So that when the healthcheck runs it is able to pull the base url and tls from docker. My eyes are going a bit squire trying to fix this so I'll have a test in the morning.

Ah ok I see

This thread @Kevin Kho

Did you mean another thread?

Sorry, yes https://prefect-community.slack.com/archives/CL09KU1K7/p1630515171385000?thread_ts=1630515171.385000&cid=CL09KU1K7

Ah ok. I didn’t read through that a while ago. What is your RunConfiguration? and the is the image of that? I think using a base image like

prefecthq/prefect:python.3.6

would pull that 3.6.14

For the run config we use

ECSRun()

and pull images from an ECR repo. I have just noticed that the CI logs do show:

FROM prefecthq/prefect:0.15.4-python3.6

when building the image. How do I control that @Kevin Kho?

We don’t have versions for every Python version so you would likely need your own image. Maybe you can try making a

pipenv

and activating it? Conda is hard to activate, pipenv might be easier but i’m not sure.