https://prefect.io logo
Channels
Powered by
m

Matt Allen

09/02/2020, 7:02 PM
Has anyone had any luck running the build and register step for their flows in a gitlab CI runner? I'm having trouble with the docker storage being unable to find the docker API when running in gitlab
c

Chris White

09/02/2020, 7:05 PM
Hi Matt, when building / registering from CircleCI, the following Docker config was useful for me:
Copy code
import docker

default_client = docker.from_env()

storage = Docker(
        base_url=default_client.api.base_url,
        tls_config=docker.TLSConfig(default_client.api.cert),
        ...,
)
m

Matt Allen

09/02/2020, 7:07 PM
I'll give that a shot, thanks
👍 1
Did you need any extra options in your gitlabci.yaml file as well?
c

Chris White

09/02/2020, 7:08 PM
This was in GitHub actually
m

Matt Allen

09/02/2020, 7:09 PM
Ah gotcha
c

Chris White

09/02/2020, 7:09 PM
but no additional config necessary in GitHub or Circle FWIW
👍 1
m

Matt Allen

09/02/2020, 8:15 PM
Do you know if there's a way to pass in private registry credentials to the Docker storage object?
c

Chris White

09/02/2020, 8:24 PM
You can’t pass them in via the Python API, but the 
docker
client itself uses whatever auth your local docker daemon has; so as long as you authenticate from the docker CLI you should be good
m

Matt Allen

09/02/2020, 8:27 PM
Ah, I see. Thanks
👍 1
a

Andy Dyer

10/01/2020, 3:30 PM
@Matt Allen @Chris White do you have any more insight into this. Trying to do docker in docker to register a flow on gitlab ci and even with these changes still getting issues
Copy code
File "/usr/local/lib/python3.8/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
m

Matt Allen

10/01/2020, 3:32 PM
Do you have this variable in your gitlabci file? 
DOCKER_HOST: <tcp://docker:2376>
Along with
Copy code
services:
  - docker:19.03.12-dind
a

Andy Dyer

10/01/2020, 3:34 PM
i have the dind but not that var. Ill try it thanks
new issue, this is my first docker in docker so I appreciate the help
Copy code
raise DockerException(
docker.errors.DockerException: Error while fetching server API version: HTTPConnectionPool(host='docker', port=2376): Max retries exceeded with url: /version (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fc4c4bafee0>: Failed to establish a new connection: [Errno 111] Connection refused'))
m

Matt Allen

10/01/2020, 3:45 PM
Are you running docker login before you run your register flow script?
I actually switched away from using the docker storage because of speed issues, so I'm trying to remember my debugging steps
a

Andy Dyer

10/01/2020, 3:46 PM
right so im not using docker storage. I am using GCP container registry
its probably that i just need to gcloud auth first isnt it? a connection refused error is so opaque
m

Matt Allen

10/01/2020, 3:54 PM
Ah yeah that could be it. If you're not using docker storage I don't think you'd need docker in docker at all actually
a

Andy Dyer

10/01/2020, 4:15 PM
yeah everything is authenticated still getting the same error. Certainly you need docker in docker right? because i am in a docker container (gitlab ci) building the docker storage for a flow to register and send off to gcr.
m

Matt Allen

10/01/2020, 4:36 PM
Oh I was confused when you said you weren't using docker storage, nevermind
The error you are getting still looks like it can't connect to the docker daemon, which happens before it tries to authenticate with gcp container registry
8 Views
Powered by