https://prefect.io logo
Join Slack
Powered by
<@U01QEJ9PP53> When mounting a volume using the pr...
# ask-community
s
@Kevin Kho When mounting a volume using the prefect docker agent, it all seems to work, but the docker container flows don't see anything in the mounted volume:
Copy code
prefect agent docker start --token <token>--name "EMG-DCK-01" --volume company:/mnt/company --label production --log-level DEBUG --show-flow-logs
Any flow using this agent then shows that directory as empty. When I manually run the container (on the same account, terminal windows, etc as the agent) and mount it with -v using the same syntax it tests out fine and the volume has the expected files:
Copy code
docker run -it -v company:/mnt/company node:lts bash

cd /mnt/company
ls

**shows all files**
Any ideas why the volume when mounted with the agent configuration would not be working? This is what the company volume configuration looks like:
Copy code
PS C:\Users\azure.automation> docker volume inspect company
[
    {
        "CreatedAt": "2021-07-17T22:43:31Z",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/company/_data",
        "Name": "company",
        "Options": {
            "device": "//192.168.1.2/Company",
            "o": "user=myusername,domain=mydomain,password=mypassword",
            "type": "cifs"
        },
        "Scope": "local"
    }
]
I think this issue might be related to a docker bind mount vs a volume. Your language in the documentation isn't totally clear, but does mention using bind mounts. I'm guessing that means you can only access the local filesystem with this then? Why does it limit it to bind mounts?
k
Hey @Seth Coussens, I'll look into this tomorrow
🙏 1
z
Hey @Seth Coussens -- we attempt to parse what you've provided to either: create a bind mount, create a named volume, or link to an existing named volume. I'm not sure why this isn't working though, the logic is complicated for win32. Can you share the generated config for a flow run container?
s
Sure, how do I best share the config? Just the Dockerfile?
Here is the flow configuration that I'm using just trying to test this out:
Copy code
import os

import numpy
import prefect
from prefect import Flow, task
from prefect.run_configs import DockerRun, UniversalRun
from prefect.storage import Docker, GitHub

from utilities import reverse_str


@task
def list_dir(x):
    # return [f for f in os.listdir(x) if not os.path.isfile(f)]
    logger = prefect.context.get('logger')
    <http://logger.info|logger.info>(f'Listing directory: {x}')
    ls = os.listdir(x)
    <http://logger.info|logger.info>(ls)


@task
def display(x):
    logger = prefect.context.get('logger')
    <http://logger.info|logger.info>(x)


@task
def sum(x):
    return numpy.sum(x)


with Flow("example") as flow:
    x = reverse_str('Hello World')
    y = sum([6, 2])
    z = list_dir('/mnt/company')

    b = display(x)
    c = display(y)

flow.storage = Docker(registry_url="<http://emgcr.azurecr.io/prefect/demo|emgcr.azurecr.io/prefect/demo>", base_image="prefecthq/prefect:latest-python3.8", files={
    "/source/emg-prefect/utilities": "/utilities"
}, env_vars={
    "PYTHONPATH": "$PYTHONPATH:utilities/"
}, python_dependencies=["numpy"])

flow.register(project_name="demo", labels=["production"])
As you can see, for this test example I'm just using the prefect base image and then just trying to log out the contents of the mounted directory.
I'm more than willing to get on a call too and share my screen to walk through this if that helps.
z
Ah sorry I want to see the information for the container that is created for a run with something like 
docker inspect
s
@Zanie
Copy code
PS D:\Source> docker image inspect <http://emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00|emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00>
[
    {
        "Id": "sha256:8ebd8d5e0a8bdd2e25c4369af7dfe68e0c7b5db278c2649634c3ea062167fbb3",
        "RepoTags": [
            "<http://emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00|emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00>"
        ],
        "RepoDigests": [
            "<http://emgcr.azurecr.io/prefect/demo/example@sha256:39a84da6c189812f08cb557b40bd2d78a8e962d2dbdec25a6a4a4c575cadcb11|emgcr.azurecr.io/prefect/demo/example@sha256:39a84da6c189812f08cb557b40bd2d78a8e962d2dbdec25a6a4a4c575cadcb11>"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2021-07-19T00:19:32.3698524Z",
        "Container": "87ebeb038b9c33175ee5b1af582394679691ce39ff377dbcf0f1a38c4b3e8100",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568",
                "PYTHON_VERSION=3.8.11",
                "PYTHON_PIP_VERSION=21.1.3",
                "PYTHON_GET_PIP_URL=<https://github.com/pypa/get-pip/raw/a1675ab6c2bd898ed82b1f58c486097f763c74a9/public/get-pip.py>",
                "PYTHON_GET_PIP_SHA256=6665659241292b2147b58922b9ffe11dda66b39d52d8a6f3aa310bc1d60ea6f7",
                "LC_ALL=C.UTF-8",
                "PYTHONPATH=$PYTHONPATH:utilities/",
                "PREFECT__USER_CONFIG_PATH=/opt/prefect/config.toml"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "python /opt/prefect/healthcheck.py '[\"/opt/prefect/flows/example.prefect\"]' '(3, 8)'"
            ],
            "Image": "sha256:4f50382a24481d8211bbf4ec14823773aabb2b5ffc19b095767f84bbd66d428d",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": [],
            "Labels": {
                "io.prefect.python-version": "3.8.11",
                "maintainer": "<mailto:help@prefect.io|help@prefect.io>",
                "org.label-schema.build-date": "2021-07-12T22:25:15Z",
                "org.label-schema.name": "prefect",
                "org.label-schema.schema-version": "= 1.0",
                "org.label-schema.url": "<https://www.prefect.io/>",
                "org.label-schema.vcs-ref": "c7218bd049841ce7a48ab78ffa9d334b200d5e68",
                "org.label-schema.version": "0.15.1"
            }
        },
        "DockerVersion": "20.10.7",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568",
                "PYTHON_VERSION=3.8.11",
                "PYTHON_PIP_VERSION=21.1.3",
                "PYTHON_GET_PIP_URL=<https://github.com/pypa/get-pip/raw/a1675ab6c2bd898ed82b1f58c486097f763c74a9/public/get-pip.py>",
                "PYTHON_GET_PIP_SHA256=6665659241292b2147b58922b9ffe11dda66b39d52d8a6f3aa310bc1d60ea6f7",
                "LC_ALL=C.UTF-8",
                "PYTHONPATH=$PYTHONPATH:utilities/",
                "PREFECT__USER_CONFIG_PATH=/opt/prefect/config.toml"
            ],
            "Cmd": null,
            "Image": "sha256:4f50382a24481d8211bbf4ec14823773aabb2b5ffc19b095767f84bbd66d428d",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "tini",
                "-g",
                "--",
                "entrypoint.sh"
            ],
            "OnBuild": [],
            "Labels": {
                "io.prefect.python-version": "3.8.11",
                "maintainer": "<mailto:help@prefect.io|help@prefect.io>",
                "org.label-schema.build-date": "2021-07-12T22:25:15Z",
                "org.label-schema.name": "prefect",
                "org.label-schema.schema-version": "= 1.0",
                "org.label-schema.url": "<https://www.prefect.io/>",
                "org.label-schema.vcs-ref": "c7218bd049841ce7a48ab78ffa9d334b200d5e68",
                "org.label-schema.version": "0.15.1"
            }
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 516070116,
        "VirtualSize": 516070116,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/dad1ab4036e4e93d29657ed2474661034f6bfafbf5dfda09db5d88ed06f1d302/diff:/var/lib/docker/overlay2/185233dafeae0aa0ee74edb7bf22a669d81a271745eab25cec0ab2d163fb60fc/diff:/var/lib/docker/overlay2/564da112eed80a64c965fafce5cc1e88fa0c8983f82aad5308e310abea02ef61/diff:/var/lib/docker/overlay2/08f374dc8f2051064b6e2fd34d24f98644c64296eea4edabbb24c874f64f0c21/diff:/var/lib/docker/overlay2/16c0510ae360f225956405ba983bb702e0cf8351d258349dc7595421ee471662/diff:/var/lib/docker/overlay2/337080d42d508f501f3fc3d18a5e24334c1861d06bba5a1cd8618c13f9d36883/diff:/var/lib/docker/overlay2/393b40dc224e55be95f27d7c1095421230191a0ea3e6d9d6bbe7d4a0bd3f7483/diff:/var/lib/docker/overlay2/2c73fbd37a662cff981db2596ffcd7930d129ba4ab474c2e88036ff45a6cb66f/diff:/var/lib/docker/overlay2/0f2d7d863e4d2afe0dc3fb3a60e35d91941c48ae0da6d643da9908763b8afcae/diff:/var/lib/docker/overlay2/9958554de689ae45b8352a295425d42664baa793399bbeca9b14cb5af88480af/diff:/var/lib/docker/overlay2/eeeb66827d3434036f49139fa0a4def8e3a1e96d4ea794779b8741ac0c79774a/diff:/var/lib/docker/overlay2/b0d4f253c4ecb2740950f10f060f24199b962e69bf861e0a4c10d7c2e3de9843/diff:/var/lib/docker/overlay2/2757c93e368e224cbb0c10a1a9c5599ab92776bfc553891d56cd498664e8e0bf/diff",
                "MergedDir": "/var/lib/docker/overlay2/9d3c06768578157a87e0ec41ba4f1a76bcf53fcee94a0f46ad3893fadf33603d/merged",
                "UpperDir": "/var/lib/docker/overlay2/9d3c06768578157a87e0ec41ba4f1a76bcf53fcee94a0f46ad3893fadf33603d/diff",
                "WorkDir": "/var/lib/docker/overlay2/9d3c06768578157a87e0ec41ba4f1a76bcf53fcee94a0f46ad3893fadf33603d/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:764055ebc9a7a290b64d17cf9ea550f1099c202d83795aa967428ebdf335c9f7",
                "sha256:537313a13d9080fc30444461b8aac5f03b9cbb0e2352b3c3098d5eca5f42688f",
                "sha256:2138286d3aea914658cda630abbc7c2b8d2f89220ac98f92181393da307a1959",
                "sha256:8dd62b771c62257bd906b2adac7b47dbf70c8062d436ee7f827db4ae9f5d6cdf",
                "sha256:abbd1e0c6d6b400268044022875460767bee31458a59cae2a2bf3b9f7c90a250",
                "sha256:3c0227b8ec967a232b9a747d1cc36c9959ba0254f63b79aa406b23de2e3b92f5",
                "sha256:010788bf76c36cfe691ef8438c28a8ad1175e447bfc320c92b19aebbf2542fd1",
                "sha256:7364efbdb174d799af00aff13d9eb2f094f95a5eecd2fd80522a99fa9a165ba6",
                "sha256:2c41d4c1cc92e40818e8215a21fa8cbed9e7d61c9335c604ba9b5673a86e24ae",
                "sha256:72c60f52b539d15a5880f2dbd985cebdf19d2c4cfa8b18cfe8f9029fbbcb464b",
                "sha256:8951a2f4246220a512790312264b6ed74255599a0c2af25f8417b73eec358f14",
                "sha256:f62383a8d3cdc1f37978c91895d1c2faf56c6a3b1fa18f3921c5894bcaadab72",
                "sha256:dff0cc6eda5760f6e32a0587ddea30e9ecfdbd4b7cda4474d2c0094876c0b633",
                "sha256:04f88f6260123f7023054448ca21d04720752d67e963c4cad2f5a890af82f666"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]
@Zanie this is one that was created by the agent when I submitted the run, I pulled the inspect while the flow was running:
Copy code
[
    {
        "Id": "7d0a3d8d25045359106ee452b11fd520971363041d79abc7a00c334df5cf992e",
        "Created": "2021-07-19T21:01:40.9263036Z",
        "Path": "tini",
        "Args": [
            "-g",
            "--",
            "entrypoint.sh",
            "prefect",
            "execute",
            "flow-run"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 10485,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2021-07-19T21:01:41.8520303Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:8ebd8d5e0a8bdd2e25c4369af7dfe68e0c7b5db278c2649634c3ea062167fbb3",
        "ResolvConfPath": "/var/lib/docker/containers/7d0a3d8d25045359106ee452b11fd520971363041d79abc7a00c334df5cf992e/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/7d0a3d8d25045359106ee452b11fd520971363041d79abc7a00c334df5cf992e/hostname",
        "HostsPath": "/var/lib/docker/containers/7d0a3d8d25045359106ee452b11fd520971363041d79abc7a00c334df5cf992e/hosts",
        "LogPath": "/var/lib/docker/containers/7d0a3d8d25045359106ee452b11fd520971363041d79abc7a00c334df5cf992e/7d0a3d8d25045359106ee452b11fd520971363041d79abc7a00c334df5cf992e-json.log",
        "Name": "/lemon-viper",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": null,
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": true,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/945697b0672751e1083cd739f3677d7dc5e3bbb72161dcd3a963d0918505055a-init/diff:/var/lib/docker/overlay2/f73def77646c585a994a2ad5784273aabcf17bd2998a4daf8a8b5931eb7dea02/diff:/var/lib/docker/overlay2/7af32021528f706189f0f90f7244d8984b2e1a86a63011fea3a522ea6bd3c07a/diff:/var/lib/docker/overlay2/a224eef3785a9f465222a3faef9c3d4b24310716fbc424020aee6fc3c3dff6c2/diff:/var/lib/docker/overlay2/c9a58180540ed7be6e945f5415ebec0b923108f14890ce16c63ee8280e102665/diff:/var/lib/docker/overlay2/3626421e3ccdb7275232cd62f9108f7cdf0c87d392c36dd615c2cb9b637d0e42/diff:/var/lib/docker/overlay2/4c71fb42c15abb6bcf6746e33cb611c9ae883f595fbbc37c1a981ba248c6a605/diff:/var/lib/docker/overlay2/7d0a9688e5368d2d6e74fdfc1d6b52ba9f543a0a10ab3f2766abfe1d6e9cac7e/diff:/var/lib/docker/overlay2/2bf24c071f56874bb07321f62d3bd20300141155e0b61f0bdaf8596404360e66/diff:/var/lib/docker/overlay2/a20e476178095888ac4a1661fe83d09d12080c3f6a3696a0cdb62fe5efc304f1/diff:/var/lib/docker/overlay2/c71ce47e1af9372aac99756fcd25469c990997ad9b8746c326204fe45ffc672b/diff:/var/lib/docker/overlay2/aa45c3ea10f5e534c85ac36ec8ef3411b3980aff1f8b8529fca5e272076e1387/diff:/var/lib/docker/overlay2/e7b881a59af315454f50c381b3e3189b28c2cf67b601d58684379632f05459a7/diff:/var/lib/docker/overlay2/a0047c036c1a4040893ddd0fb4d082107c31cc3ba863da8d7ff656cd8144436b/diff:/var/lib/docker/overlay2/90806fae7f3f43061ce8dbe479e7fbea665f84bd566ab5e5fef831fdf8c43777/diff",
                "MergedDir": "/var/lib/docker/overlay2/945697b0672751e1083cd739f3677d7dc5e3bbb72161dcd3a963d0918505055a/merged",
                "UpperDir": "/var/lib/docker/overlay2/945697b0672751e1083cd739f3677d7dc5e3bbb72161dcd3a963d0918505055a/diff",
                "WorkDir": "/var/lib/docker/overlay2/945697b0672751e1083cd739f3677d7dc5e3bbb72161dcd3a963d0918505055a/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "c807c9e4c2cae9dcd311a578508693d5bd03d7a5703da285b7b88f136b2b2e40",
                "Source": "/var/lib/docker/volumes/c807c9e4c2cae9dcd311a578508693d5bd03d7a5703da285b7b88f136b2b2e40/_data",
                "Destination": "/mnt/company",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "7d0a3d8d2504",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PREFECT__LOGGING__LEVEL=INFO",
                "PREFECT__BACKEND=cloud",
                "PREFECT__CLOUD__API=<https://api.prefect.io>",
                "PREFECT__CLOUD__AUTH_TOKEN=kSDwdDhq1Zs1h6OiJ0nRgw",
                "PREFECT__CLOUD__AGENT__LABELS=['production']",
                "PREFECT__CLOUD__SEND_FLOW_RUN_LOGS=true",
                "PREFECT__CONTEXT__FLOW_RUN_ID=e1f3649f-e141-4e98-8f46-d10954b0c52c",
                "PREFECT__CONTEXT__FLOW_ID=5ea1bfb9-4e3a-4440-87cd-dacd489cfec3",
                "PREFECT__CONTEXT__IMAGE=<http://emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00|emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00>",
                "PREFECT__CLOUD__USE_LOCAL_SECRETS=false",
                "PREFECT__ENGINE__FLOW_RUNNER__DEFAULT_CLASS=prefect.engine.cloud.CloudFlowRunner",
                "PREFECT__ENGINE__TASK_RUNNER__DEFAULT_CLASS=prefect.engine.cloud.CloudTaskRunner",
                "PREFECT__LOGGING__LOG_TO_CLOUD=true",
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568",
                "PYTHON_VERSION=3.8.11",
                "PYTHON_PIP_VERSION=21.1.3",
                "PYTHON_GET_PIP_URL=<https://github.com/pypa/get-pip/raw/a1675ab6c2bd898ed82b1f58c486097f763c74a9/public/get-pip.py>",
                "PYTHON_GET_PIP_SHA256=6665659241292b2147b58922b9ffe11dda66b39d52d8a6f3aa310bc1d60ea6f7",
                "LC_ALL=C.UTF-8",
                "PYTHONPATH=$PYTHONPATH:utilities/",
                "PREFECT__USER_CONFIG_PATH=/opt/prefect/config.toml"
            ],
            "Cmd": [
                "prefect",
                "execute",
                "flow-run"
            ],
            "Image": "<http://emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00|emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00>",
            "Volumes": {
                "/mnt/company": {}
            },
            "WorkingDir": "",
            "Entrypoint": [
                "tini",
                "-g",
                "--",
                "entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "io.prefect.flow-id": "5ea1bfb9-4e3a-4440-87cd-dacd489cfec3",
                "io.prefect.flow-name": "example",
                "io.prefect.flow-run-id": "e1f3649f-e141-4e98-8f46-d10954b0c52c",
                "io.prefect.python-version": "3.8.11",
                "maintainer": "<mailto:help@prefect.io|help@prefect.io>",
                "org.label-schema.build-date": "2021-07-12T22:25:15Z",
                "org.label-schema.name": "prefect",
                "org.label-schema.schema-version": "= 1.0",
                "org.label-schema.url": "<https://www.prefect.io/>",
                "org.label-schema.vcs-ref": "c7218bd049841ce7a48ab78ffa9d334b200d5e68",
                "org.label-schema.version": "0.15.1"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "09f4083432094e2a4955374091bca41f04b810714c0c312c13731943b1578d4a",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/09f408343209",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "10475edfb99c047d81f0e5e9946d22e6e32cf1289a23293e6ebfffb21777a494",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "59e152f07952747e0756649d272e2c3fe113cc14d6d92c56962ffd2090962178",
                    "EndpointID": "10475edfb99c047d81f0e5e9946d22e6e32cf1289a23293e6ebfffb21777a494",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]
@Zanie for contrast, here is the inspect for a similar image, but run manually with the following command:
Copy code
docker run -it -v company:/mnt/company <http://emgcr.azurecr.io/prefect/demo/example:2021-07-1|emgcr.azurecr.io/prefect/demo/example:2021-07-1>
9t00-19-08-479168-00-00 bash
Here is the inspect information for that one (hint, it shows a mount but no volume, where as the other shows a mount and volume, but they dont seem to match):
Copy code
[
    {
        "Id": "e80354a12a6e0dc9446ea8001b13155dbd0d89939842a29dfcaae1c30a1bf372",
        "Created": "2021-07-19T21:03:50.3973156Z",
        "Path": "tini",
        "Args": [
            "-g",
            "--",
            "entrypoint.sh",
            "bash"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 10592,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2021-07-19T21:03:51.1375655Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:8ebd8d5e0a8bdd2e25c4369af7dfe68e0c7b5db278c2649634c3ea062167fbb3",
        "ResolvConfPath": "/var/lib/docker/containers/e80354a12a6e0dc9446ea8001b13155dbd0d89939842a29dfcaae1c30a1bf372/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/e80354a12a6e0dc9446ea8001b13155dbd0d89939842a29dfcaae1c30a1bf372/hostname",
        "HostsPath": "/var/lib/docker/containers/e80354a12a6e0dc9446ea8001b13155dbd0d89939842a29dfcaae1c30a1bf372/hosts",
        "LogPath": "/var/lib/docker/containers/e80354a12a6e0dc9446ea8001b13155dbd0d89939842a29dfcaae1c30a1bf372/e80354a12a6e0dc9446ea8001b13155dbd0d89939842a29dfcaae1c30a1bf372-json.log",
        "Name": "/gifted_rhodes",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "company:/mnt/company"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                37,
                116
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/86218c2047da8dcf51f25b7c30ee8a454a6ea0602ac6b733cb13d5d39afd8468-init/diff:/var/lib/docker/overlay2/f73def77646c585a994a2ad5784273aabcf17bd2998a4daf8a8b5931eb7dea02/diff:/var/lib/docker/overlay2/7af32021528f706189f0f90f7244d8984b2e1a86a63011fea3a522ea6bd3c07a/diff:/var/lib/docker/overlay2/a224eef3785a9f465222a3faef9c3d4b24310716fbc424020aee6fc3c3dff6c2/diff:/var/lib/docker/overlay2/c9a58180540ed7be6e945f5415ebec0b923108f14890ce16c63ee8280e102665/diff:/var/lib/docker/overlay2/3626421e3ccdb7275232cd62f9108f7cdf0c87d392c36dd615c2cb9b637d0e42/diff:/var/lib/docker/overlay2/4c71fb42c15abb6bcf6746e33cb611c9ae883f595fbbc37c1a981ba248c6a605/diff:/var/lib/docker/overlay2/7d0a9688e5368d2d6e74fdfc1d6b52ba9f543a0a10ab3f2766abfe1d6e9cac7e/diff:/var/lib/docker/overlay2/2bf24c071f56874bb07321f62d3bd20300141155e0b61f0bdaf8596404360e66/diff:/var/lib/docker/overlay2/a20e476178095888ac4a1661fe83d09d12080c3f6a3696a0cdb62fe5efc304f1/diff:/var/lib/docker/overlay2/c71ce47e1af9372aac99756fcd25469c990997ad9b8746c326204fe45ffc672b/diff:/var/lib/docker/overlay2/aa45c3ea10f5e534c85ac36ec8ef3411b3980aff1f8b8529fca5e272076e1387/diff:/var/lib/docker/overlay2/e7b881a59af315454f50c381b3e3189b28c2cf67b601d58684379632f05459a7/diff:/var/lib/docker/overlay2/a0047c036c1a4040893ddd0fb4d082107c31cc3ba863da8d7ff656cd8144436b/diff:/var/lib/docker/overlay2/90806fae7f3f43061ce8dbe479e7fbea665f84bd566ab5e5fef831fdf8c43777/diff",
                "MergedDir": "/var/lib/docker/overlay2/86218c2047da8dcf51f25b7c30ee8a454a6ea0602ac6b733cb13d5d39afd8468/merged",
                "UpperDir": "/var/lib/docker/overlay2/86218c2047da8dcf51f25b7c30ee8a454a6ea0602ac6b733cb13d5d39afd8468/diff",
                "WorkDir": "/var/lib/docker/overlay2/86218c2047da8dcf51f25b7c30ee8a454a6ea0602ac6b733cb13d5d39afd8468/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "company",
                "Source": "/var/lib/docker/volumes/company/_data",
                "Destination": "/mnt/company",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "e80354a12a6e",
            "Domainname": "",
            "User": "",
            "AttachStdin": true,
            "AttachStdout": true,
            "AttachStderr": true,
            "Tty": true,
            "OpenStdin": true,
            "StdinOnce": true,
            "Env": [
                "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=C.UTF-8",
                "GPG_KEY=E3FF2839C048B25C084DEBE9B26995E310250568",
                "PYTHON_VERSION=3.8.11",
                "PYTHON_PIP_VERSION=21.1.3",
                "PYTHON_GET_PIP_URL=<https://github.com/pypa/get-pip/raw/a1675ab6c2bd898ed82b1f58c486097f763c74a9/public/get-pip.py>",
                "PYTHON_GET_PIP_SHA256=6665659241292b2147b58922b9ffe11dda66b39d52d8a6f3aa310bc1d60ea6f7",
                "LC_ALL=C.UTF-8",
                "PYTHONPATH=$PYTHONPATH:utilities/",
                "PREFECT__USER_CONFIG_PATH=/opt/prefect/config.toml"
            ],
            "Cmd": [
                "bash"
            ],
            "Image": "<http://emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00|emgcr.azurecr.io/prefect/demo/example:2021-07-19t00-19-08-479168-00-00>",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "tini",
                "-g",
                "--",
                "entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "io.prefect.python-version": "3.8.11",
                "maintainer": "<mailto:help@prefect.io|help@prefect.io>",
                "org.label-schema.build-date": "2021-07-12T22:25:15Z",
                "org.label-schema.name": "prefect",
                "org.label-schema.schema-version": "= 1.0",
                "org.label-schema.url": "<https://www.prefect.io/>",
                "org.label-schema.vcs-ref": "c7218bd049841ce7a48ab78ffa9d334b200d5e68",
                "org.label-schema.version": "0.15.1"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "9a20841d0718f47c47b5c88ace68afbb79b96049dbe78e4f21d7f81ae3418324",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/9a20841d0718",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "e63ec743905d2cf7820f76b0f1be6223bd99582c6020b7229010b9d9a4a0b8f6",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "59e152f07952747e0756649d272e2c3fe113cc14d6d92c56962ffd2090962178",
                    "EndpointID": "e63ec743905d2cf7820f76b0f1be6223bd99582c6020b7229010b9d9a4a0b8f6",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]
@Zanie any updates?
z
Yeah so the differences here appear to be the lack of 
Binds
, a lack of 
Mounts.Mode
, and a different 
Mounts.Source
location. We'd need to try adding some of this information to the run call in the 
DockerAgent
, I think 
Binds
is the mostly likely candidate for the missing data you're seeing.
s
Long story short though, not something I can fix at the moment?
z
It doesn't look like it at a glance unless you want to give a try at contributing. Generally, it's hard to reproduce issues on Windows/Docker. In our release today we'll be exposing the 
host_config
directly in the 
DockerRun
so you could easily override the binds.
Here's a simple change that I think should always include the binds in the host_config if you want to give it a try, but dockerpy doesn't always mirror the Docker API as you'd expect: https://github.com/PrefectHQ/prefect/compare/fixup-docker
s
@Zanie I made those changes to the agent.py to test it out and it looks like it works. 👍🏻
I would think the same change would be needed on the linux side too? Not necessary for us though.
4 Views
Open in Slack
PreviousNext
Powered by