https://prefect.io logo
Join Slack
Powered by
Hello again! Yes I’m back :sweat_smile: The proble...
# ask-community
a
Hello again! Yes I’m back 😅 The problem: Pointing to the private registry (a 
DockerRegistry
block) through the 
image_registry
parameter of 
DockerContainer
doesn’t work, I can tell that by looking at the Docker registry field in the GUI is still empty. This means I have to manually go to the GUI and point it to the registry. I don’t want to set it myself each time, how do I accomplish this through the library?
✅ 1
n
hi @Ajeel Ahmed - can you share what you're trying programmatically that you're saying has no effect?
hmm okay I've reproduced this, i think this is a UI problem - when I load the saved block programmatically, it still has the registry attached
a
it does not update yes, so when I try to run the flow when the docker registry field is empty it crashes so I have to go to GUI update the field myself and then run the flow and then it does work
I hope you get what I’m trying to say haha
n
when the docker registry field is empty it crashes so I have to go to GUI update the field myself
this is surprising to me, because i see that the registry stays attached to the 
DockerContainer
(below) I would expect that when the agent pulls the 
DockerContainer
, that it still has the 
image_registry
filled out, even if the UI doesn't show it for some reason
Copy code
In [5]: DockerContainer(image_registry=registry)
Out[5]: DockerContainer(type='docker-container', env={}, labels={}, name=None, command=None, image='prefecthq/prefect:2.10.8-python3.10', image_pull_policy=None, image_registry=DockerRegistry(username='zzstoatzz', password=SecretStr('**********'), registry_url='<http://registry.hub.docker.com|registry.hub.docker.com>', reauth=True), networks=[], network_mode=None, auto_remove=False, volumes=[], stream_output=True, memswap_limit=None, mem_limit=None, privileged=False)

In [6]: DockerContainer(image_registry=registry).save("test")
Out[6]: UUID('e49c230d-5017-4000-aae3-85346b55124f')

In [7]: exit

❯ ipython

In [1]: from prefect.infrastructure.docker  import DockerContainer, DockerRegistry

In [2]: DockerContainer.load("test")
Out[2]: DockerContainer(type='docker-container', env={}, labels={}, name=None, command=None, image='prefecthq/prefect:2.10.8-python3.10', image_pull_policy=None, image_registry=DockerRegistry(username='zzstoatzz', password=SecretStr('**********'), registry_url='<http://registry.hub.docker.com|registry.hub.docker.com>', reauth=True), networks=[], network_mode=None, auto_remove=False, volumes=[], stream_output=True, memswap_limit=None, mem_limit=None, privileged=False)
a
Copy code
registry_block = DockerRegistry.load("gitlab-docker-registry")

    infra_block = DockerContainer(
        image_registry=registry_block,
        image="link to the image",
        image_pull_policy="ALWAYS",
        networks=[network stuff]
        env=asdict(Config()),
        auto_remove=True,
        stream_output=True,
    )
this is the code
here’s the thing: the documentation says the image parameter should take in the tag of the docker image, does that mean I’m supposed to only put the tag name? Like 
latest
? If so where am I supposed to put the link to the image?
you can see it here
n
im pretty sure that docstring is slightly misleading and 
image
wants something like 
registry/image:tag
can you share the logs from when it crashes?
a
yeah sure just give me a sec
Copy code
2023-05-18 22:46:03 21:46:03.494 | DEBUG   | prefect.agent - Checking for cancelled flow runs...
2023-05-18 22:46:13 21:46:13.368 | DEBUG   | prefect.agent - Checking for cancelled flow runs...
2023-05-18 22:46:16 21:46:16.759 | DEBUG   | prefect.agent - Checking for scheduled flow runs...
2023-05-18 22:46:27 21:46:27.000 | DEBUG   | prefect.agent - Checking for scheduled flow runs...
2023-05-18 22:46:27 21:46:27.184 | DEBUG   | prefect.agent - Checking for cancelled flow runs...
2023-05-18 22:46:35 21:46:35.382 | DEBUG   | prefect.agent - Checking for scheduled flow runs...
2023-05-18 22:46:35 21:46:35.864 | INFO    | prefect.agent - Submitting flow run '4e3de005-6c51-492d-8bf7-437c6088167c'
2023-05-18 22:46:36 21:46:36.954 | DEBUG   | prefect.infrastructure.docker-container - Could not find Docker image locally: <http://registry.gitlab.com/planet-farms/periplo-innovation/project-precog/ml_db:latest|registry.gitlab.com/planet-farms/periplo-innovation/project-precog/ml_db:latest>
2023-05-18 22:46:36 21:46:36.958 | INFO    | prefect.infrastructure.docker-container - Pulling image '<http://registry.gitlab.com/planet-farms/periplo-innovation/project-precog/ml_db:latest|registry.gitlab.com/planet-farms/periplo-innovation/project-precog/ml_db:latest>'...
2023-05-18 22:46:38 21:46:37.980 | ERROR   | prefect.agent - Failed to submit flow run '4e3de005-6c51-492d-8bf7-437c6088167c' to infrastructure.
2023-05-18 22:46:38 Traceback (most recent call last):
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/docker/api/client.py", line 268, in _raise_for_status
2023-05-18 22:46:38     response.raise_for_status()
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
2023-05-18 22:46:38     raise HTTPError(http_error_msg, response=self)
2023-05-18 22:46:38 requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: <http+docker://localhost/v1.41/images/create?tag=latest&fromImage=registry.gitlab.com%2Fplanet-farms%2Fperiplo-innovation%2Fproject-precog%2Fml_db>
2023-05-18 22:46:38 
2023-05-18 22:46:38 The above exception was the direct cause of the following exception:
2023-05-18 22:46:38 
2023-05-18 22:46:38 Traceback (most recent call last):
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/prefect/agent.py", line 424, in _submit_run_and_capture_errors
2023-05-18 22:46:38     result = await infrastructure.run(task_status=task_status)
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/prefect/infrastructure/docker.py", line 267, in run
2023-05-18 22:46:38     container = await run_sync_in_worker_thread(self._create_and_start_container)
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 91, in run_sync_in_worker_thread
2023-05-18 22:46:38     return await anyio.to_thread.run_sync(
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
2023-05-18 22:46:38     return await get_asynclib().run_sync_in_worker_thread(
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
2023-05-18 22:46:38     return await future
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
2023-05-18 22:46:38     result = context.run(func, *args)
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/prefect/infrastructure/docker.py", line 365, in _create_and_start_container
2023-05-18 22:46:38     self._pull_image(docker_client)
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/prefect/infrastructure/docker.py", line 475, in _pull_image
2023-05-18 22:46:38     return docker_client.images.pull(image, tag)
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/docker/models/images.py", line 465, in pull
2023-05-18 22:46:38     pull_log = self.client.api.pull(
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/docker/api/image.py", line 429, in pull
2023-05-18 22:46:38     self._raise_for_status(response)
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/docker/api/client.py", line 270, in _raise_for_status
2023-05-18 22:46:38     raise create_api_error_from_http_exception(e) from e
2023-05-18 22:46:38   File "/usr/local/lib/python3.10/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
2023-05-18 22:46:38     raise cls(e, response=response, explanation=explanation) from e
2023-05-18 22:46:38 docker.errors.APIError: 500 Server Error for <http+docker://localhost/v1.41/images/create?tag=latest&fromImage=registry.gitlab.com%2Fplanet-farms%2Fperiplo-innovation%2Fproject-precog%2Fml_db>: Internal Server Error ("Head "<https://registry.gitlab.com/v2/planet-farms/periplo-innovation/project-precog/ml_db/manifests/latest>": denied: access forbidden")
this is the error when the field for the Docker Registry is empty
so when I fill it with the Docker Registry block it’s supposed to have (the one that is in the code above) and run the flow again it runs just fine
n
hmm this seems like a bug then. would you be willing to open an issue?
a
oh yeah of course! I was talking to a coworker and I was sure we were doing something wrong but good to know this is probably a bug!
n
i could be missing something as well, but its at least a little weird, since the registry block should definitely appear in the UI after saving the block with the registry attached
a
yeah we can close it if it’s something on my side
n
👍
🙌 1
a
Hey @Nate hope you had a great weekend, I thought I’d investigate this issue a bit more before making an issue on GitHub and fortunately we realised the mistake, our setup for blocks was failing and we weren’t getting notified so it was not updating the blocks that were already there, which made me think there was something wrong. Fixed now, Prefect works just fine! Thanks again.
🙌 1
n
thanks for digging into that and letting me know 🙌 appreciate the update - glad things are working!
🙌 1
4 Views
Open in Slack
PreviousNext
Powered by