Hello! I'm stuck on something that should be simple, but can't find answer in channel history or docs! I am using prefect.storage.Docker for flow storage inside Docker image. I would like to push this image up to Dockerhub. When I try that using the code below, I get "InterruptedError: unauthorized: authentication required" error. I am able to successfully login to Dockerhub using Docker SDK, but not sure how to pass this authenticated client to prefect.storage.Docker. And I am not sure how I authenticate within prefect.storage.Docker. As suggested by the docs, I am running the Docker daemon locally, and logged into docker in the terminal both of which are configured to push up to the repository. How can I fix? Thanks!

from prefect import task, Flow
from prefect.executors import LocalExecutor
from prefect.run_configs import DockerRun
from prefect.storage import Docker
import docker

with Flow("some_flow") as flow:
    do_something

    docker_client = docker.DockerClient()
    docker_client.login(username=<env_user>,password=<env_pass>)

    flow.storage = Docker(
        registry_url='<http://registry.hub.docker.com/repository/docker/<user>/<repo>|registry.hub.docker.com/repository/docker/<user>/<repo>>'
        ,image_name='<some_flow>'
        ,files={
            <origin path>:<dest path>
        }
        ,python_dependencies = ['pandas','numpy','prefect']
        ,env_vars={
            "PYTHONPATH": "$PYTHONPATH:assets/:root/:data/:image"
        }
        ,base_image='python:3.7.3'
    )

    flow.run_config = DockerRun(labels=['my-label']
                                )
    flow.executor = LocalExecutor()
    flow.register(project_name="some_project")

There are a couple of issues here: 1. Did you authenticate with your registry before you tried to register your flow? Usually you should do something like “docker login -u username -p password” in the terminal rather than doing it within your flow 2. Starting from storage your indentation seems to be off - you can either do one of those:

with Flow("some_flow") as flow:
    do_something

    docker_client = docker.DockerClient()
    docker_client.login(username=<env_user>,password=<env_pass>)

flow.storage = Docker(
    registry_url='<http://registry.hub.docker.com/repository/docker/<user>/<repo>|registry.hub.docker.com/repository/docker/<user>/<repo>>'
    ,image_name='<some_flow>'
    ,files={
           <origin path>:<dest path>
}
,python_dependencies = ['pandas','numpy','prefect']
,env_vars={
              "PYTHONPATH": "$PYTHONPATH:assets/:root/:data/:image"
          }
,base_image='python:3.7.3'
)

flow.run_config = DockerRun(labels=['my-label']
                            )
flow.executor = LocalExecutor()
flow.register(project_name="some_project")

or:

with Flow(
    "some_flow",
    run_config=DockerRun(labels=["my-label"]),
    storage=Docker(
        registry_url="<http://registry.hub.docker.com/repository/docker/<user>/<repo>|registry.hub.docker.com/repository/docker/<user>/<repo>>",
        image_name="<some_flow>",
        python_dependencies=["pandas", "numpy", "prefect"],
        env_vars={"PYTHONPATH": "$PYTHONPATH:assets/:root/:data/:image"},
        base_image="python:3.7.3",
    ),
) as flow:
    pass

3. Lastly, within your Flow constructor, you can only construct a DAG, i.e. you can’t have any non-task code. Within the “with Flow” block you should only be calling tasks, so you can remove the lines below from your Flow block and replace that with login via terminal in the same terminal session in which you register your flow:

docker_client = docker.DockerClient()
   docker_client.login(username=<env_user>,password=<env_pass>)

Anna, Thanks for the quick reply. Here's what I know: 1. Yes, I authenticated with my registry with "docker login -u <user> -p <password>" before running "python flow.py" in the same terminal. I received a "Login Succeeded" after authenticating. 2. My sincere apologies, this was my error in copying/pasting while sanitizing the code 3. Same here, I had the docker_client lines commented out in my actual code (was just testing to see if it would log in) Tried "docker login" again to doublecheck, but still getting the "authentication required" 😕

can you try to build and push the image to registry without Prefect and check whether this works?

Thanks for that suggestion -- it sent me down a rabbit hole, but ultimately realized that it worked if I used the repository name as "image_name" and the image name as "image_tag"; working code looks as follows (and fixed the mistakes you highlighted above):

from prefect import task, Flow
from prefect.executors import LocalExecutor
from prefect.run_configs import DockerRun
from prefect.storage import Docker
import docker

with Flow("some_flow") as flow:
    do_something

flow.storage = Docker(
        registry_url='<user>'
        ,image_name='<repo>'
        ,image_tag='<some_flow>'
        ,files={
            <origin path>:<dest path>
        }
        ,python_dependencies = ['pandas','numpy','prefect']
        ,env_vars={
            "PYTHONPATH": "$PYTHONPATH:assets/:root/:data/:image"
        }
        ,base_image='python:3.7.3'
    )

    flow.run_config = DockerRun(labels=['my-label']
                                )
    flow.executor = LocalExecutor()
    flow.register(project_name="some_project")

nice!