Hi all, hope you're having a lovely day. I'm having some trouble with docker storage on CircleCI. Hoping someone can help me debug it. When calling

Docker(registry_url=...)

I get the following exception:

docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', BadStatusLine('\x15\x03\x01\x00\x02\x02\n'))

Any ideas why that is? FWIW, I'm using Google Container Registry and I've already authenticated with it (running

docker pull <http://gcr.io/etc/etc/myprivateimage|gcr.io/etc/etc/myprivateimage>

works fine). CircleCI does use a 'remote docker' to run such commands so perhaps this command isn't able to connect to it?

Hi @Adam, hope you're having a lovely day as well! I think you're correct about this being related to Circle's remote docker setup. If so, setting

DOCKER_HOST

as discussed in this thread may help you out. https://prefect-community.slack.com/archives/CL09KU1K7/p1584128047269800

Thanks @Zachary Hughes. This thread is indeed on topic! Unfortunately I'm still have an issue after using the DOCKER_HOST env var (FWIW, that seems to now be natively supported by prefect: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/environments/storage/docker.py#L136)

Okay, solid. After setting

DOCKER_HOST

, are you still seeing the same error, or is it a different one?

I've upgraded the version of Docker to 19.x and I'm now getting an exception about it trying to connect to an HTTPS server using HTTP. I'm now working on setting

tls_config

parameter, hopefully that helps

Great, new errors generally mean progress! Let me know if there's anything I can do to help here.

So, I think I've managed to get it all fixed 🙂 Is there an easy way in Prefect Cloud to see the latest docker container associated with a flow? For reference purposes, for anyone else using CircleCI. This is what worked: In `.circleci/config.yaml`:

- setup_remote_docker:
          version: 19.03.12
          docker_layer_caching: true

And within your code:

tls_config = docker.tls.TLSConfig(
    client_cert=(
        path.join(environ.get("DOCKER_CERT_PATH"), "cert.pem"),
        path.join(environ.get("DOCKER_CERT_PATH"), "key.pem"),
    )
)

flow.storage = Docker(
    registry_url="<http://gcr.io/xxx/yyy|gcr.io/xxx/yyy>",
    base_url=environ.get("DOCKER_HOST"),
    tls_config=tls_config,
)
flow.register(project_name="prefect-test-1")

Awesome! If you want to find storage information about a specific flow, you can run the following query:

query {
  flow (where: {id: {_eq: "YOUR-FLOW-ID-HERE"}}){
    id
    storage
  }
}

and if you want to get it for a flow group, the first result from this query will do the trick for you:

query {
  flow (where: {flow_group_id: {_eq: "YOUR-FLOW-GROUP-ID-HERE"}}, order_by: {created: desc}){
    id
    storage
  }
}

Brilliant! Thanks @Zachary Hughes 😄

You got it! Let us know if there's anything else we can do to help.

@Marvin archive “Docker storage in CircleCI using TLS”

i’m having this exact issue in Google Cloud Build right now

This just saved me, thanks all! 🙏