Joe Schmid
03/13/2020, 7:34 PMdocker run hello-world
succeeds. (There is special "docker-in-docker" config we use with Screwdriver and this works well for many of our other projects.) I can also build Docker storage successfully on my laptop from inside the same image that our CI/CD system is using. I'm sure it's some simple config that I'm missing -- anybody run into this?
15:25:50 Traceback (most recent call last):
15:25:50 File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
15:25:50 "__main__", mod_spec)
15:25:50 File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
15:25:50 exec(code, run_globals)
15:25:50 File "/usr/local/lib/python3.7/site-packages/srm_datasci-0.1.0-py3.7.egg/srm/datasci/flows/util/test_flow.py", line 35, in <module>
15:25:50 flow.register("Utilities")
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/core/flow.py", line 1412, in register
15:25:50 no_url=no_url,
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 616, in register
15:25:50 serialized_flow = flow.serialize(build=build) # type: Any
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/core/flow.py", line 1221, in serialize
15:25:50 storage = self.storage.build() # type: Optional[Storage]
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 263, in build
15:25:50 self._build_image(push=push)
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 293, in _build_image
15:25:50 self.pull_image()
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 483, in pull_image
15:25:50 client = self._get_client()
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 471, in _get_client
15:25:50 return docker.APIClient(base_url=self.base_url, version="auto")
15:25:50 File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 190, in __init__
15:25:50 self._version = self._retrieve_server_version()
15:25:50 File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 215, in _retrieve_server_version
15:25:50 'Error while fetching server API version: {0}'.format(e)
15:25:50 docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
<tcp://localhost:2375>
So for us the change was:
from prefect.environments.storage import Docker
storage = Docker(
registry_url="<redacted>",
image_name="<redacted>",
image_tag=IMAGE_TAG,
python_dependencies=["<redacted>"],
base_url=environ.get("DOCKER_HOST"),
)
It might be good to bubble this change down into Prefect, e.g:
default_url = environ.get("DOCKER_HOST", "<unix://var/run/docker.sock>")
Here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/environments/storage/docker.py#L96Laura Lorenz (she/her)
03/13/2020, 8:49 PMJoe Schmid
03/13/2020, 8:53 PMLaura Lorenz (she/her)
03/13/2020, 8:54 PMMarvin
03/13/2020, 9:40 PM