Thanks for the details — this sounds like a volume bind path issue on Windows. Key points: - In a Prefect 3.x Docker work pool, the

volumes

you set in the Base Job template are passed straight through to the Docker API. Prefect does not translate paths. - On Windows with Docker Desktop, you should use Windows-style host paths (e.g.,

C:\...

) on the host side of the bind, and Unix-style paths on the container side (e.g.,

/database

). Using

/host_mnt/c/...

typically won’t work from Windows because Docker Desktop expects

C:\...

and does the translation internally. What to change - Update your work pool’s Base Job template volumes from:

[
    "/host_mnt/c/qfin/database:/database:rw"
  ]

to:

[
    "C:\\qfin\\database:/database:rw"
  ]

Note: In JSON (UI editor), you must escape backslashes, so each backslash becomes

\\

. - Your env var looks good and should remain:

{
  "QFIN_DATABASE_DUCK": "/database/database.duckdb"
}

Other things to check - Docker Desktop drive sharing: ensure the C: drive is shared/allowed for file sharing in Docker Desktop settings. If it’s not, binds will fail. - Permissions: some images run as a non-root user. If you need write access, confirm the container user can write to

/database

(you’ve set

:rw

, which is good). - Confirm the template you think is active is actually set: - You can inspect the pool to see the current base job template:

prefect work-pool inspect "<your-pool-name>"

- If needed, fetch the default template for comparison:

prefect work-pool get-default-base-job-template --type docker

Quick validation - After updating the volume string, run a tiny flow that lists

/database

to confirm the mount is present:

from prefect import flow
  import os

  @flow
  def check_mount():
      print("Contents of /database:", os.listdir("/database"))

  if __name__ == "__main__":
      check_mount()

A couple of clarifying questions to make sure we’re covering all bases: - Which Prefect version are you on? (I’m assuming 3.x.) - How is your Docker worker running — as a local Python process on Windows, or inside a Docker container? - Do the worker logs show an “invalid volume specification” or “bind source path does not exist” error when creating the flow-run container? If you share the worker logs for a failed run, I can pinpoint the exact failure.

Thanks — that error helps pinpoint it. Because your Docker worker is running inside a Linux container, it’s talking to a Linux Docker daemon (Docker Desktop’s Linux VM). In that context, Windows-style host paths like

C:\qfin\database

are not valid; the daemon expects a Linux-style path that maps to your Windows drive inside the Docker Desktop VM. Use one of these Linux-style host paths instead: - Preferred on recent Docker Desktop:

[
    "/run/desktop/mnt/host/c/qfin/database:/database:rw"
  ]

- Older/alias path (often still works):

[
    "/host_mnt/c/qfin/database:/database:rw"
  ]

Keep your env var as-is:

{
  "QFIN_DATABASE_DUCK": "/database/database.duckdb"
}

Why this changed - When the worker runs in a container, Docker won’t translate

C:\...

paths. That translation only happens when the client is Windows-native. The Linux daemon rejects

C:\...:/...

with “invalid volume specification,” which is exactly what you’re seeing. Quick checks - Validate which Linux path works on your machine: - From your host, run:

docker run --rm -v /run/desktop/mnt/host/c/qfin/database:/database alpine ls -la /database

If that fails, try:

docker run --rm -v /host_mnt/c/qfin/database:/database alpine ls -la /database

- Ensure Docker Desktop Settings > Resources > File sharing includes

C:\qfin\database

. - Make sure volumes are set under

job_configuration.volumes

in the work pool’s Base Job template (not under

container_create_kwargs

— the worker ignores volumes there). If you prefer using the CLI to confirm/update the template: - Inspect the current template:

prefect work-pool inspect "<your-pool-name>"

- Get the default Docker template (for reference):

prefect work-pool get-default-base-job-template --type docker

- Update the pool with a modified JSON file:

prefect work-pool update "<your-pool-name>" --base-job-template ./template.json

If you switch back to a Linux-style host path and still can’t see the files inside

/database

, share a snippet of the worker logs from the container creation step and I’ll dig in further.

Great—those docker run tests are super helpful. On your machine, /host_mnt/c/... is the correct host path (since it shows your database.duckdb). So the bind path itself is fine. Given that: - docker run sees the file via /host_mnt/c/qfin/database - Your flow-run containers can’t The most common cause is that the deployment’s job variables are overriding the work pool Base Job template volumes. In Prefect 3.x: - volumes are NOT merged; deployment job_variables.volumes replaces the base template - if a deployment sets volumes: [] or omits volumes while the base job template uses "{{ volumes }}", you’ll end up with no mounts - volumes under container_create_kwargs are ignored by the Docker worker Quick checks 1) Inspect your deployment to see what it’s actually passing:

prefect deployment inspect "<flow-name>/<deployment-name>"

Look for job_variables.volumes. If it’s empty or missing, that explains it. 2) Inspect your work pool to confirm what your Base Job template currently has:

prefect work-pool inspect "<your-pool-name>"

- If job_configuration.volumes is "{{ volumes }}", then you must provide volumes in each deployment’s job_variables. - If you changed job_configuration.volumes to a concrete list, that should be used unless a deployment overrides it. What to set - Since /host_mnt works on your system, use:

[
  "/host_mnt/c/qfin/database:/database:rw"
]

- Keep your env var:

{
  "QFIN_DATABASE_DUCK": "/database/database.duckdb"
}

Two reliable ways to apply it - Set volumes and env directly in the deployment’s job_variables (recommended so it’s tied to the deployment): - In code when deploying: job_variables={"volumes": ["/host_mnt/c/qfin/database/databaserw"], "env": {"QFIN_DATABASE_DUCK": "/database/database.duckdb"}} - Or update via UI/CLI and redeploy. - Alternatively, make the Base Job template use concrete values (not "{{ volumes }}" / "{{ env }}") under job_configuration, and ensure deployments do not set job_variables.volumes (or they’ll override it). Diagnose inside the flow-run container - Add a quick flow run to verify both the mount and env:

from prefect import flow
import os, sys

@flow
def debug_mount():
    print("env QFIN_DATABASE_DUCK:", os.environ.get("QFIN_DATABASE_DUCK"))
    try:
        print("ls /database:", os.listdir("/database"))
    except Exception as e:
        print("ls error:", e, file=sys.stderr)

if __name__ == "__main__":
    debug_mount()

Extra visibility (optional) - Run the Docker worker with DEBUG logs to see the final container settings it uses: - Set PREFECT_LOGGING_LEVEL=DEBUG on the worker container and check logs around container creation; you should see the volumes list. Note on File sharing - Your docker run test proves the share is working for /host_mnt/c/qfin/database even without explicitly configuring File sharing in Docker Desktop, so that’s not the blocker here. If after setting the volumes in the deployment you still don’t see the file, please paste: - prefect deployment inspect output for job_variables - the Docker worker logs around container creation for that run and I’ll help pinpoint the exact override.

Thanks — that context helps a lot. Here’s what your data tells us: - docker run proves /host_mnt/c/qfin/database works on your machine (it lists database.duckdb), while /run/desktop/mnt/host/... does not. So /host_mnt/c/... is the right host path for your Docker Desktop setup. - Your deployment shows no job_variables.volumes, and your work pool’s Base Job template has 'volumes': '{{ volumes }}' with no concrete default in job_configuration. In Prefect 3.x, volumes do not merge; they are replaced. If nothing provides a value for the {{ volumes }} placeholder, the final container gets no volumes. Why env is working but volumes are not - env has special deep-merge behavior and defaults pulled from multiple places; even with '{{ env }}', the worker adds base Prefect env, flow-run env, etc., so you still see env vars. - volumes have no such merge/default behavior. If your base template says '{{ volumes }}' and your deployment/job variables do not provide volumes, nothing gets mounted. Two ways to fix Pick one, then re-run a flow to confirm. Option A: Set volumes at the deployment level (recommended so it’s explicit per deployment) - Add job_variables to the deployment with your working mount: - UI: Deployment > Edit > Job variables - JSON example:

{
  "volumes": [
    "/host_mnt/c/qfin/database:/database:rw"
  ],
  "env": {
    "QFIN_DATABASE_DUCK": "/database/database.duckdb"
  }
}

- Re-run the flow and verify the file is found. Option B: Provide a default in the Base Job template variables schema - Edit the work pool Base Job template so variables.properties.volumes has a default:

"variables": {
    "properties": {
      "volumes": {
        "type": "array",
        "items": {"type": "string"},
        "default": ["/host_mnt/c/qfin/database:/database:rw"]
      },
      "env": {
        "type": "object",
        "default": {"QFIN_DATABASE_DUCK": "/database/database.duckdb"}
      }
    }
  },
  "job_configuration": {
    "env": "{{ env }}",
    "volumes": "{{ volumes }}"
  }

- This way, even if deployments don’t specify volumes, the default will be applied. How to get the Docker worker logs around container creation - If you’re running the Prefect Docker worker in a container: - Find the container name/id:

docker ps | grep prefect

(or by label if you set one) - Tail the logs:

docker logs -f <worker-container>

- For more detail, set PREFECT_LOGGING_LEVEL=DEBUG on the worker container. You can add this env var to the worker’s container and restart it. Then watch logs again; you should see the final container settings, including volumes, passed to Docker for each flow run. Quick verification flow - After setting volumes as above, run a simple flow to confirm mount + env:

from prefect import flow
  import os, sys

  @flow
  def debug_mount():
      print("QFIN_DATABASE_DUCK:", os.environ.get("QFIN_DATABASE_DUCK"))
      try:
          print("ls /database:", os.listdir("/database"))
      except Exception as e:
          print("ls error:", e, file=sys.stderr)

  if __name__ == "__main__":
      debug_mount()

If you prefer, share: - The relevant part of your work pool Base Job template (variables.properties and job_configuration sections), and - The output of prefect deployment inspect <flow>/<deployment> showing job_variables and I’ll suggest the exact JSON to paste so mounts are guaranteed for all runs.

Got it — thanks for confirming. Since you don’t want to set per-deployment volumes and this used to work, I suspect the issue is where the default is defined in your Base Job template. Key nuance in Prefect 3.x: - env is deep-merged and gets defaults from multiple places, so you’ll see env vars even if the base job template uses '{{ env }}'. - volumes are not merged; they’re replaced. To apply a default across all deployments, the default must be defined exactly under variables.properties.volumes.default. If the default is placed under configuration.properties.volumes.default (a common pitfall), it won’t be applied for normal flow runs. What I’d like you to double-check Please paste the relevant parts from: - prefect work-pool inspect "<your-pool-name>" Specifically these sections: - job_configuration (confirm it shows)

"env": "{{ env }}",
  "volumes": "{{ volumes }}"

- variables.properties.volumes (confirm the default lives here) Example of the correct spot:

"variables": {
    "properties": {
      "volumes": {
        "type": "array",
        "items": {"type": "string"},
        "default": ["/host_mnt/c/qfin/database:/database:rw"]
      },
      "env": {
        "type": "object",
        "default": {"QFIN_DATABASE_DUCK": "/database/database.duckdb"}
      }
    }
  }

If your defaults are under configuration.properties.* they won’t be picked up by normal flow runs. Quick forcing test (to confirm the template is the problem) Set a concrete value in job_configuration.volumes (not a placeholder) just to verify volumes get mounted across all deployments:

"job_configuration": {
  "volumes": ["/host_mnt/c/qfin/database:/database:rw"],
  "env": "{{ env }}"
}

If that works, it confirms the defaults were just defined in the wrong section. You can decide whether to keep it concrete (simple) or move the default to variables.properties.volumes.default. How to see what the flow-run container actually got Since you have the container name in the error earlier (e.g., faithful-pogona), after a run starts you can inspect: - Binds:

docker inspect faithful-pogona --format '{{json .HostConfig.Binds}}'

- Mounts:

docker inspect faithful-pogona --format '{{json .Mounts}}'

If the volumes default isn’t being applied, Binds will be null/[]. Other checks - Ensure your deployment really targets this work pool/queue (not another pool with a different base template). - Confirm the volume string is an array of strings exactly (not a single string or nested array). - Continue using /host_mnt/c/qfin/database on your system (your docker run tests confirm that path works). If you share the “variables” block (especially where the volumes default is defined) from prefect work-pool inspect, I can give you the exact JSON patch to fix it.

If the binds and mounts show up in docker inspect for the flow-run container, Prefect is successfully passing the volume. At that point, the issue is almost certainly inside the container (path mismatch, env var content, permissions, or DuckDB opening the file). Let’s verify inside the actual flow-run container. 1) Exec into the running flow-run container and check the mount, env var, and ability to open the file:

# Replace <container_id_or_name> with your flow-run container
docker exec -it <container_id_or_name> sh -lc '
set -e
echo "Dest mounts:"; mount | grep "on /database " || true
echo "Env var:"; python - << "PY"
import os, reprlib
v = os.environ.get("QFIN_DATABASE_DUCK")
print("QFIN_DATABASE_DUCK repr:", repr(v))
PY
echo "List /database:"; ls -la /database || true
echo "Python checks:"; python - << "PY"
import os, sys
p = os.environ.get("QFIN_DATABASE_DUCK")
print("Exists:", os.path.exists(p), "Path:", repr(p))
try:
    import duckdb
    con = duckdb.connect(p)
    print("DuckDB connect OK")
    con.close()
except Exception as e:
    print("DuckDB error:", e, file=sys.stderr)
PY
'

What this will tell us: - mount | grep shows that /database is actually mounted. - repr of QFIN_DATABASE_DUCK will reveal any invisible whitespace or stray characters. - ls -la /database confirms the file is visible in that container. - duckdb.connect(p) will show if DuckDB can open it or return a specific error (file not found vs locked vs permissions). 2) Double-check the container user If your image runs as a non-root user, confirm the UID/GID and permissions:

docker exec -it <container_id_or_name> sh -lc 'id && ls -ld /database && ls -l /database/database.duckdb'

Your earlier ls output suggested the file is 0644 and directory 0755, which should be readable by any user; but let’s confirm in the actual flow-run image. 3) Confirm no trailing whitespace/newline in env var It’s surprisingly common from UI edits. The repr() check above will expose things like '/database/database.duckdb\n'. 4) Confirm the exact path matches the mount - Destination in docker inspect should be exactly /database. - The env var should be /database/database.duckdb (matching your earlier config). If everything above checks out, please share: - The output of those docker exec commands (especially the repr of QFIN_DATABASE_DUCK and the DuckDB error) - The output of:

docker inspect <container_id_or_name> --format '{{json .HostConfig.Binds}}'
  docker inspect <container_id_or_name> --format '{{json .Mounts}}'

- The exact error/stack trace your code prints when it “can’t find” the file Given that mounts are present, it’s unlikely a Prefect template issue at this point. The most common culprits I’ve seen in this situation are: - A tiny mismatch in the env var value (e.g., trailing space or wrong filename) - Running as a non-root user with insufficient directory execute permissions (rare in your case, but worth confirming) - The image’s entrypoint working directory or code resolving a different path than expected (the absolute path avoids this, but still worth testing with os.path.exists and duckdb.connect in-container) If you prefer not to exec manually, you can also bake a minimal Prefect flow that prints repr(os.environ["QFIN_DATABASE_DUCK"]), lists /database, and tries duckdb.connect — but the docker exec method above is the fastest to isolate the problem.

Yep — if Binds and Mounts show the /database mount in docker inspect for the flow-run container, Prefect is successfully passing the volume. At that point, the failure is happening inside the container when DuckDB tries to open the file. Likely culprits and what to do next: 1) Get the exact DuckDB error from the flow-run container logs - Since the container crashes, grab the logs after it exits:

docker logs attentive-anaconda

Please paste the exact DuckDB error (e.g., “No such file or directory”, “Permission denied”, “Could not set lock”). 2) Quick manual repro using your image Run your image with the same mount and try opening the DB. This isolates the image/user/permissions from Prefect:

docker run --rm \
  -v /host_mnt/c/qfin/database:/database \
  -e QFIN_DATABASE_DUCK=/database/database.duckdb \
  <your-flow-image> sh -lc '
    id
    ls -la /database
    python - << "PY"
import os, duckdb, sys
p = os.environ.get("QFIN_DATABASE_DUCK")
print("repr:", repr(p))
print("exists:", os.path.exists(p))
try:
    duckdb.connect(p).close()
    print("connect OK")
except Exception as e:
    print("error:", e, file=sys.stderr)
PY'

- If this fails with “Permission denied” or “Could not set lock”: your container user likely lacks write perms on /database. By default the directory is 755 (owner-write only) and the file may be 644; many DuckDB operations need to write a lock/WAL next to the DB even if you only read. Options: - If you only need reads: open read-only and make the mount read-only: - In code:

duckdb.connect(os.environ["QFIN_DATABASE_DUCK"], read_only=True)

- In volumes:

"/host_mnt/c/qfin/database:/database:ro"

- If you need writes: run the container as root or a user that can write that directory: - Add a default user to your work pool Base Job template via container_create_kwargs so it applies to all deployments:

{
  "variables": {
    "properties": {
      "container_create_kwargs": {
        "type": "object",
        "default": {
          "user": "0:0"
        }
      },
      "volumes": {
        "type": "array",
        "items": {
          "type": "string"
        },
        "default": [
          "/host_mnt/c/qfin/database:/database:rw"
        ]
      },
      "env": {
        "type": "object",
        "default": {
          "QFIN_DATABASE_DUCK": "/database/database.duckdb"
        }
      }
    }
  },
  "job_configuration": {
    "container_create_kwargs": "{{ container_create_kwargs }}",
    "volumes": "{{ volumes }}",
    "env": "{{ env }}"
  }
}

Note: volumes in container_create_kwargs are ignored by the Docker worker, but user is forwarded to Docker. - If the manual test succeeds but Prefect runs still fail, we need the exact error from docker logs for the flow-run container to see what’s different at runtime. 3) If you can tolerate a small code change, add pre-checks before connecting so the container doesn’t crash immediately and logs useful info:

import os, sys, duckdb
p = os.environ.get("QFIN_DATABASE_DUCK")
print("QFIN_DATABASE_DUCK repr:", repr(p))
try:
    print("ls /database:", os.listdir("/database"))
except Exception as e:
    print("ls error:", e, file=sys.stderr)
# then connect...

Key points aligned with your goals: - You do not need to set volumes per deployment. Keeping the default under variables.properties.volumes.default in the work pool Base Job template is the right approach. - Since Binds/Mounts show the /database mount, Prefect is doing its part; we just need the exact DuckDB error to choose between read-only open vs running as root vs adjusting permissions. Share the output of: - docker logs attentive-anaconda - The manual docker run test above and I’ll recommend the precise fix.

Good news: since Binds and Mounts show the /database mount on the flow‑run container, Prefect is definitely passing the volume. We need to pinpoint what’s different at runtime inside that container. Most likely causes now: - The image used by the flow‑run isn’t the one you expect (different Python/duckdb/env). - The env var value inside that run isn’t what you think (trailing space/quotes, different key, or not set). - The code is opening a different path than you expect, or the env var is being overridden. Let’s verify with docker inspect/logs for the failed flow‑run container. Please run these on the flow‑run container that failed (e.g., attentive-anaconda): 1) Confirm the exact image used

docker inspect attentive-anaconda --format '{{.Config.Image}}'

2) Dump the container env and check QFIN_DATABASE_DUCK precisely

docker inspect attentive-anaconda --format '{{json .Config.Env}}'

- Look for an entry like

QFIN_DATABASE_DUCK=/database/database.duckdb

- This reveals invisible whitespace or quoting issues. 3) Get the logs (to see the exact exception text from os.path.exists or duckdb)

docker logs attentive-anaconda

Your manual test showed: - Same bind works - os.path.exists('/database/database.duckdb') == True …so the image + env + mount combination can work. The difference must be one of: - The flow‑run used a different image than brrr/runtime:dev - QFIN_DATABASE_DUCK is different at runtime (or not set) - The code path uses a different variable or default path when env is missing A few quick guardrails you can apply now: - Make the “exists” flow log repr of the env var and list the directory before connecting, so it won’t crash:

import os, sys
  p = os.environ.get("QFIN_DATABASE_DUCK")
  print("QFIN_DATABASE_DUCK repr:", repr(p))
  try:
      print("ls /database:", os.listdir("/database"))
  except Exception as e:
      print("ls error:", e, file=sys.stderr)
  print("exists:", os.path.exists(p))

- Temporarily prevent DuckDB from silently creating a new DB by opening read-only and/or making the mount read-only: - In code:

duckdb.connect(os.environ["QFIN_DATABASE_DUCK"], read_only=True)

- In volumes:

"/host_mnt/c/qfin/database:/database:ro"

This will fail loudly if the path isn’t exactly right. If it turns out to be a user/image mismatch: - You can enforce a specific user or image across all deployments via the Base Job template defaults (so you don’t set it per deployment):

{
    "variables": {
      "properties": {
        "image": {
          "type": "string",
          "default": "brrr/runtime:dev"
        },
        "env": {
          "type": "object",
          "default": {"QFIN_DATABASE_DUCK": "/database/database.duckdb"}
        },
        "volumes": {
          "type": "array",
          "items": {"type": "string"},
          "default": ["/host_mnt/c/qfin/database:/database:rw"]
        },
        "container_create_kwargs": {
          "type": "object",
          "default": {"user": "0:0"}  // only if you need root
        }
      }
    },
    "job_configuration": {
      "image": "{{ image }}",
      "env": "{{ env }}",
      "volumes": "{{ volumes }}",
      "container_create_kwargs": "{{ container_create_kwargs }}"
    }
  }

Next step - Please share: - docker inspect attentive-anaconda --format '{{.Config.Image}}' - docker inspect attentive-anaconda --format '{{json .Config.Env}}' (just the QFIN_DATABASE_DUCK bit) - docker logs attentive-anaconda With those three, we’ll know exactly whether it’s the image, the env value, or something else causing os.path.exists to return False despite the bind being present.

Thanks — that output narrows it down. What we know: - The flow-run container has the bind mount: /host_mnt/c/qfin/database -> /database - Inside that container, ls /database shows only database.duckdb; dev_database.duckdb is not there - Your manual docker run using the same image and mount reproduces this: exists('/database/dev_database.duckdb') == False Conclusion: the file dev_database.duckdb is not visible to Docker Desktop’s Linux VM at that path. Prefect is passing the volume correctly; this is a host-filesystem visibility issue. Common causes on Windows and how to check/fix: 1) File is cloud-only (OneDrive/Dropbox “online-only”) - Cloud placeholders are not present on disk and won’t show up in the bind mount. - Fix: In File Explorer, right-click dev_database.duckdb → “Always keep on this device”. Wait for it to fully download. - Re-run:

docker run --rm -v /host_mnt/c/qfin/database:/database alpine ls -la /database

You should now see dev_database.duckdb. 2) It’s not actually in C:\qfin\database - Double-check in an elevated PowerShell on the host:

dir C:\qfin\database

Confirm the exact name and that it’s in that folder (not a different path). 3) It’s a shortcut/junction/symlink - A Windows shortcut (.lnk) or a symlink/junction may not resolve inside the container. - Check attributes:

attrib "C:\qfin\database\dev_database.duckdb"

If it’s a reparse point/symlink, copy the real file into C:\qfin\database as a normal file. 4) Case or invisible characters in the filename - Windows is case-insensitive; Linux is not. Ensure the exact case matches dev_database.duckdb. - Also check for trailing spaces or odd characters. In PowerShell:

dir C:\qfin\database | Format-List Name,Length,LastWriteTime

Sanity checks you can run: - Create a sentinel file on Windows and see it from the container: - On Windows: create C:\qfin\database\_probe.txt - Then:

docker run --rm -v /host_mnt/c/qfin/database:/database alpine ls -la /database

You should see _probe.txt. - Or create a file from inside the container and see it on Windows:

docker run --rm -v /host_mnt/c/qfin/database:/database alpine sh -lc 'echo hello > /database/_from_container.txt; ls -la /database'

Then check C:\qfin\database on Windows for _from_container.txt. Once dev_database.duckdb is truly on-disk in C:\qfin\database, your existing setup should work: - Work pool Base Job template default: - volumes: ["/host_mnt/c/qfin/database/databaserw"] - env: {"QFIN_DATABASE_DUCK": "/database/dev_database.duckdb"} - Your flows will see the file without per-deployment config. Optional hardening: - If the DB is read-only for flows, make it explicit to avoid accidental creation: - volumes: ["/host_mnt/c/qfin/database/databasero"] - In code: duckdb.connect(os.environ["QFIN_DATABASE_DUCK"], read_only=True) If after ensuring the file is local and not a reparse/shortcut you still don’t see it inside the container, share: - PowerShell dir output of C:\qfin\database showing dev_database.duckdb - The alpine ls -la /database output after that and we’ll dig deeper.

That discrepancy is the smoking gun: you’re hitting two different Docker daemons/contexts. - From PowerShell, your docker run sees both big files (database.duckdb and dev_database.duckdb) under /host_mnt/c/... — that’s the Docker Desktop daemon view. - From Ubuntu/WSL, the same docker run only sees a tiny 12 KB database.duckdb — that’s almost certainly a different daemon (e.g., a Docker engine running inside WSL), where /host_mnt/c/... is just a local directory in that daemon’s VM, not your Windows drive. If your Prefect Docker worker container is talking to the WSL daemon, it will create flow-run containers against that daemon — which won’t see the real Windows files. That exactly matches your flow-run behavior. What to do 1) Align your docker context in WSL to Docker Desktop - In WSL: - Check contexts:

docker context ls

- Switch to Docker Desktop’s Linux daemon (commonly named desktop-linux):

docker context use desktop-linux

- Verify:

docker info | grep -i 'docker desktop'

- Re-test in WSL:

docker run --rm -v /host_mnt/c/qfin/database:/database alpine ls -la /database

You should now see both large files (and the big sizes). 2) Ensure your Prefect worker container talks to Docker Desktop - If you’re mounting a socket into the worker, confirm it’s Docker Desktop’s socket: - Inspect the worker container:

docker inspect <your-worker-container> --format '{{json .Mounts}}'

You should see something like a bind of /var/run/docker.sock to the host’s Docker Desktop socket. - If you accidentally bind-mounted a WSL Docker socket into the worker, it’ll point at the wrong daemon. Fix the bind to the Docker Desktop socket and restart the worker. 3) Keep using the same volume path in the Base Job template - Once everything points to Docker Desktop, your existing default will work:

"volumes": ["/host_mnt/c/qfin/database:/database:rw"]
  "env": {"QFIN_DATABASE_DUCK": "/database/dev_database.duckdb"}

- No need to set volumes per deployment. Optional: prefer read-only if you don’t need writes - To prevent accidental DB creation when the path doesn’t resolve: - Set the mount to ro and open DB read-only in code: - Volumes:

"/host_mnt/c/qfin/database:/database:ro"

- Code:

duckdb.connect(os.environ["QFIN_DATABASE_DUCK"], read_only=True)

If you share: - docker context ls and docker info from both PowerShell and WSL - The worker container’s Mounts (docker inspect <worker> --format '{{json .Mounts}}') I can confirm which daemon each is using and call out the exact change to make.

Bingo — you’re talking to two different Docker daemons. - In PowerShell, docker run shows both big files (database.duckdb and dev_database.duckdb) at /host_mnt/c/... — that’s the Docker Desktop daemon. - In WSL/Ubuntu, docker run initially showed only a tiny file (and after switching to desktop-linux you got “protocol not available”) — that’s because the WSL docker client was pointing at a different engine (WSL dockerd), and it can’t use the Windows npipe endpoint. You’ve got two viable paths. Pick one and keep everything consistent. Option A (recommended): Use Docker Desktop daemon from WSL and for your Prefect worker 1) Enable WSL integration in Docker Desktop - Docker Desktop > Settings > Resources > WSL Integration - Enable your Ubuntu distro - Apply & restart Docker Desktop 2) Stop any WSL Docker daemon (if installed) - In WSL:

sudo service docker stop || true
  sudo systemctl disable docker || true
  ps aux | grep -i dockerd

Ensure no WSL dockerd is running. 3) Verify the socket points to Docker Desktop - In WSL:

ls -l /var/run/docker.sock

It should be a symlink to /mnt/wsl/shared-docker/docker.sock (or exist and be owned by root:docker) 4) Use default context in WSL - The desktop-linux context (npipe) won’t work in WSL; use default:

docker context use default
  docker info | head

Now docker in WSL should talk to Docker Desktop. 5) Re-test that you see the big files from WSL

docker run --rm -v /host_mnt/c/qfin/database:/database alpine ls -la /database

You should see both dev_database.duckdb and database.duckdb with the large sizes. 6) Ensure your Prefect Docker worker is started on the Desktop engine - Start/maintain the worker from PowerShell (or from WSL after the steps above so it uses Desktop). - docker ps in PowerShell and WSL should show the same containers. Option B: Keep using a WSL Docker daemon (not recommended for your setup) - If you intentionally want to use the WSL daemon, mount the Windows path via WSL’s view instead of /host_mnt:

"/mnt/c/qfin/database:/database:rw"

- Then all docker commands (including starting the Prefect worker) must be executed against the WSL daemon. Make sure both the worker and your tests are using the same daemon. Why your earlier test failed in WSL - desktop-linux context uses npipe:// which isn’t accessible inside WSL (“protocol not available”). That’s why switching contexts broke docker run. - Once WSL is integrated with Docker Desktop, you don’t need to switch context; default will point to the Desktop daemon via a Unix socket. After alignment - Your current work pool Base Job template defaults with: - volumes: ["/host_mnt/c/qfin/database/databaserw"] - env: {"QFIN_DATABASE_DUCK": "/database/dev_database.duckdb"} should work across all deployments (no per-deployment config needed). - If the DB is read-only in flows, consider: - volumes: ["/host_mnt/c/qfin/database/databasero"] - and in code: duckdb.connect(os.environ["QFIN_DATABASE_DUCK"], read_only=True) If you want, share: - ls -l /var/run/docker.sock (from WSL) - docker info (from WSL, after integration) - docker ps (from both PowerShell and WSL) and I’ll confirm everything is pointed at the same daemon.

Right now you’re still hitting two different Docker daemons: - PowerShell → Docker Desktop daemon: you see both huge files and everything looks correct - WSL/Ubuntu → a different daemon/socket: you only see the tiny 12 KB file and miss dev_database.duckdb Your Prefect worker and flow-run containers need to talk to the same daemon as the one that can see your Windows files (Docker Desktop). Let’s align WSL to Docker Desktop and verify. Do this in order: 1) Enable WSL integration in Docker Desktop - Docker Desktop > Settings > Resources > WSL Integration - Enable your Ubuntu distro - Apply & restart Docker Desktop 2) Ensure WSL is not running its own dockerd

sudo service docker stop || true
sudo systemctl disable docker || true
ps aux | grep -i dockerd

(You already show no dockerd process — good.) 3) Make sure the WSL Docker CLI points to Docker Desktop - In WSL:

echo $DOCKER_HOST
  docker context use default
  docker version

Key: In docker version output, the Server section should say Docker Desktop or show the Desktop Engine details. If it doesn’t, Docker Desktop integration isn’t active yet. - Also check the socket:

ls -l /var/run/docker.sock

On a properly integrated setup, this socket is provided by Docker Desktop (often a link to /mnt/wsl/shared-docker/docker.sock). 4) Restart and retest - From PowerShell:

wsl --shutdown

- Restart Docker Desktop - Open WSL again, then:

docker version
  docker ps

Compare docker ps in WSL and PowerShell — they should show the same containers. 5) Verify the mount from WSL now points at Desktop

docker run --rm -v /host_mnt/c/qfin/database:/database alpine ls -la /database

You should now see both big files (database.duckdb and dev_database.duckdb) and your probe file — same as PowerShell. At that point: - Keep your work pool Base Job template defaults: - volumes: ["/host_mnt/c/qfin/database/databaserw"] - env: {"QFIN_DATABASE_DUCK": "/database/dev_database.duckdb"} - No need to set volumes per deployment. If you want me to double-check the daemon alignment, please share: - In WSL: - Output of:

echo $DOCKER_HOST

- Output of:

docker version

(full, especially Server section) - Output of:

docker context inspect default

- In PowerShell: - Output of:

docker version

- docker ps from both shells (they should match) As a temporary workaround while you align contexts, you can run/operate the Prefect worker and your tests from PowerShell (which is definitely hitting Docker Desktop) — then your flows will see the correct files immediately.

I’m really sorry this derailed your setup — let’s get you back to a stable state first, then we can return to volumes. What likely happened - You ended up with the Docker CLI in WSL pointing at a different daemon than Docker Desktop. Now containers (including Prefect’s) are failing to start or can’t resolve service names (the server’s gaierror usually means the DB service hostname can’t be resolved because that service isn’t running or networking is broken). Quick rollback to a known-good Docker Desktop 1) Fully restart Docker Desktop and WSL - PowerShell:

wsl --shutdown

- Quit Docker Desktop and start it again (wait until it’s “running”). 2) Verify Docker Desktop works from PowerShell - Run:

docker version
  docker ps
  docker run --rm hello-world
  docker run --rm -v /host_mnt/c/qfin/database:/database alpine ls -la /database

You should see both large duckdb files in the mount. 3) Verify WSL is pointed at Desktop (or avoid WSL for now) - In WSL:

echo $DOCKER_HOST
  docker context use default
  docker version
  docker ps

If Server does NOT show Docker Desktop or ps doesn’t match PowerShell, open Docker Desktop > Settings > Resources > WSL Integration and enable your Ubuntu distro, then repeat step 1 (wsl --shutdown + restart Docker Desktop). - If WSL is still flaky, run all Prefect Docker commands from PowerShell for now (workers, compose, etc.) so everything targets Docker Desktop. Bring Prefect services back up Depending on how you run Prefect Server, use the matching path: A) If you use docker compose - In the directory with your compose file:

docker compose down
  docker compose pull
  docker compose up -d
  docker compose ps
  docker compose logs -f

Ensure the database container (often named postgres or db) is up. The server’s gaierror usually means the DB service wasn’t running or resolvable. B) If you use Prefect CLI server - Stop and start the managed server:

prefect server stop
  prefect server start

Then check logs:

prefect server services logs

If you’ve set a Postgres URL, make sure the hostname resolves from the host where the server runs. Once Docker Desktop is stable - Start your Prefect Docker worker from PowerShell (so it definitely uses Desktop’s daemon). - Re-run a flow and confirm the volume: - Your Base Job template default is fine: - volumes: ["/host_mnt/c/qfin/database/databaserw"] - env: {"QFIN_DATABASE_DUCK": "/database/dev_database.duckdb"} - This avoids per-deployment config. If issues persist, please share: - How you start Prefect Server (compose file vs

prefect server start

) - The output of: - PowerShell:

docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"

- If compose:

docker compose ps

and

docker compose logs -f

for the DB and server services - In WSL and PowerShell:

docker version

(Server section) - The exact server error lines preceding the gaierror Again, apologies for the churn — once everything points to the same Docker Desktop engine, your mounts and Prefect services should behave consistently.

Exactly — that confirms it was the Docker daemon mismatch. - From PowerShell (Docker Desktop daemon), your compose + worker see /host_mnt correctly and both DuckDB files are visible. - From WSL (a different/dockerless context), the same commands hit a different environment where those files don’t exist — so Prefect flow-run containers started from there can’t see them. Recommended operating pattern - Pick a single daemon for everything and stick to it. Easiest: run all Docker/Prefect commands from PowerShell (Docker Desktop). - Start Prefect Server (docker compose) from PowerShell - Start the Docker worker from PowerShell - Build/pull images from PowerShell - If you must use WSL, ensure WSL is integrated with Docker Desktop and that

docker version

(Server section) in WSL shows the Docker Desktop engine. Do not switch to the

desktop-linux

context from WSL; use the default context with the Desktop-provided socket. With that alignment: - Keep your work pool Base Job template defaults as you had them (no per-deployment duplication needed):

"variables": {
    "properties": {
      "volumes": {
        "type": "array",
        "items": {"type": "string"},
        "default": ["/host_mnt/c/qfin/database:/database:rw"]
      },
      "env": {
        "type": "object",
        "default": {"QFIN_DATABASE_DUCK": "/database/dev_database.duckdb"}
      }
    }
  },
  "job_configuration": {
    "volumes": "{{ volumes }}",
    "env": "{{ env }}"
  }

- Your flows should now consistently see the DB file. Optional hardening - If the DB is read-only for flows: - Set the mount to

:ro

, and - Open with

duckdb.connect(os.environ["QFIN_DATABASE_DUCK"], read_only=True)

This prevents accidental DB creation if the path ever resolves incorrectly. If you ever need to verify quickly: -

docker run --rm -v /host_mnt/c/qfin/database:/database alpine ls -la /database

- Do this from the same shell you use to run Prefect — it should list both large DuckDB files. If anything gets stuck again, share the output of: - PowerShell and WSL

docker version

(Server sections) -

docker ps

from both shells (they should match) - How you started the worker (which shell), plus a snippet of worker logs at container creation and I’ll help you lock it down.