Are there any examples of running a docker agent, in a docker container with flows being run with DockerRun from script storage? I think I am having issues with the Docker-in-Docker paradigm

Hey @Rowan Gaffney , there is some info here . You need to mount the sock to run Docker in Docker

Thanks @Kevin Kho. Yep, I have the sock mounted, but when the agent starts the flow run, using DockerRun, which adds some pip packages on top of the prefecthq/prefect:core-0.15.5 (via EXTRA_PIP_PACKAGES), I get the following error:

Traceback (most recent call last):
agent_1     |   File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 169, in _new_conn
agent_1     |     conn = connection.create_connection(
agent_1     |   File "/usr/local/lib/python3.9/site-packages/urllib3/util/connection.py", line 73, in create_connection
agent_1     |     for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
agent_1     |   File "/usr/local/lib/python3.9/socket.py", line 954, in getaddrinfo
agent_1     |     for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
agent_1     | socket.gaierror: [Errno -2] Name or service not known
agent_1     | 
agent_1     | During handling of the above exception, another exception occurred:
agent_1     | 
agent_1     | Traceback (most recent call last):
agent_1     |   File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
agent_1     |     httplib_response = self._make_request(
agent_1     |   File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 394, in _make_request
agent_1     |     conn.request(method, url, **httplib_request_kw)
agent_1     |   File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 234, in request
agent_1     |     super(HTTPConnection, self).request(method, url, body=body, headers=headers)
agent_1     |   File "/usr/local/lib/python3.9/http/client.py", line 1279, in request
agent_1     |     self._send_request(method, url, body, headers, encode_chunked)
agent_1     |   File "/usr/local/lib/python3.9/http/client.py", line 1325, in _send_request
agent_1     |     self.endheaders(body, encode_chunked=encode_chunked)
agent_1     |   File "/usr/local/lib/python3.9/http/client.py", line 1274, in endheaders
agent_1     |     self._send_output(message_body, encode_chunked=encode_chunked)

Ah I see. You’re saying it’s the

pip

that can’t resolve? so I assume it works when you don’t try to install that?

ya - I think it is with pip... perhaps an issue with the resolv.conf file? I have not tried it without pip because I need to use the "prefect[gitlab]" add in. Either way, I am downloading data from an http server in a flow, so would need to have internet access within the container/flow.

That would be a good thing to try. Sorry I don’t have better idea here 😅

Will go ahead and give it a try!

Any logs on the flow side? in the UI?

only,

Submitted for execution: Container ID: c7e6599b9bce3357c604cc991acec1821698bd356ff7c18a19a2ebc8520786bb

Could you try running with debug level logs for the flow and see if anything turns up?