Thread
#best-practices-coordination-plane
    Felix Horvat

    Felix Horvat

    1 month ago
    Is is possible to have a DockerContainer Block point to a private Container registry - say on gitlab? If so how can I pass the authentication parameters? Or is the assumption that the the Docker service that an agent has access to is logged in to the required container registries?
    Anna Geller

    Anna Geller

    1 month ago
    Good question, we started building blocks for login with private registries - so far we only have one for ECR. Would you be open to contribute one for Gitlab following the same Block interface? I can help you along the way, LMK
    @Felix Horvat I take that back, I think you can already do that using this block, just set the registry to your Gitlab container registry and in theory this should "just work" - if not, it would be best to open a Github issue, you can cc me there
    Felix Horvat

    Felix Horvat

    1 month ago
    @Anna Geller I cannot find the block on my prefect version (cloud 2.0.4). And how would i add the registry to the build command? I can only pass one
    --infra-block
    Or would i pass that to the agent? To me it seems that the auth part should just be part of the docker container block - similar to how it is with s3
    Can you also point to the existing code for ECR?
    Anna Geller

    Anna Geller

    1 month ago
    It's in the main repo infrastructure/docker - sorry, on mobile now
    ben

    ben

    1 month ago
    @Felix Horvat Checking whether or not you were able to start anything related to a GitLab Docker Container block contribution. We need to use GitLab too, but can't through the DockerContainer, because we have a server port in our image names. Whatever library is working behind the scenes (docker_py, maybe?) interprets the ":" between the server and port as where the image tag begins. For example, if the image were named "gitlab.server.com:5050/repo/path/shared-image:2.1.1", it will throw an error similar to this.
    docker.errors.APIError: 400 Client Error for <http+docker://localnpipe/v1.41/images/create?tag=5050%2Frepo%2Fpath%2Fshared-image&fromImage=gitlab.server.com>: Bad Request ("invalid tag format")
    If no one is working on a contribution yet, I can give it a try as well.
    In
    prefect/infrastructure/docker.py
    When I do a proof of concept change (i.e., not production worthy)
    DockerContainer._get_image_and_tag()
    and add the noted lines, my GitLab image containing a port number (referenced above) runs successfully:
    def _get_image_and_tag(self) -> Tuple[str, Optional[str]]:
            parts = self.image.split(":")
            if len(parts) > 2:                                      # <= added
                parts = [f"{parts[0]}:{parts[1]}", parts[2]]        # <= added
            image = parts.pop(0)
            tag = parts[0] if parts else None
            return image, tag
    With the above adjustment, if I use a generic DockerContainer block together with a generic DockerRegistry block, both connected to a GitLab container registry, both seem to work fine (authenticates successfully, pulls the image, and runs it).
    I see that another person has a pull request open for this: https://github.com/PrefectHQ/prefect/pull/6477
    Anna Geller

    Anna Geller

    1 month ago
    thanks for looking into this in detail! I linked this discussion and asked the PR contributor to address the review feedback
    ben

    ben

    1 month ago
    Funny timing -- I was just coming back here to ask if there is a way that Prefect prefers for me to participate if I have a solution related to an existing PR? Since the PR contributor hadn't followed up for a couple of days I spent a few minutes last night looking at the link that was suggested by madkinsz as a source for proper parsing of a docker image string. I updated the PR author's
    parse_tag_image()
    function based on the bounds of possibilities that Docker Image Specification v1.1.0 lists.
    I know there are couple of different ways that PRs can be done against an existing PR, but I 100% don't want to step on anyone's toes -- the PR author did the work of figuring out a lot already. I just took what they did and added to it. I mainly just want to be able to use the fix as quickly as I can 😃 https://github.com/darrida/prefect/blob/cf05964fa1ff2158deffa08e70536fb1a71fa6a0/src/prefect/docker.py#L457
    Anna Geller

    Anna Geller

    1 month ago
    this looks great, could you open this as a separate PR to the Prefect repo, given that the author of the other PR didn't follow up? I think it's fine, we will give credit to you both in release notes (teamwork between contributors!) -- you can just link to the other PR and give the author credit or shoutout
    ben

    ben

    1 month ago
    That sounds great. I should be able to get PR submitted this morning (it's 9:12am where I am).