Great setup — you can absolutely use those VMs as Docker workers in a Docker work pool. Below is a simple, reliable way to wire this up in Prefect 3.x and ensure each VM can pull from your private Docker Hub registry. High-level plan - Create a Docker work pool called “Docker” - On each VM, install Prefect + the Docker worker plugin and start a worker that points to that pool - Provide credentials for your private Docker Hub registry either: - A) centrally via a Prefect DockerRegistryCredentials block (recommended), or - B) locally via

docker login

on each VM Step-by-step 1) Create the Docker work pool

prefect work-pool create "Docker" --type docker

2) Choose how to provide private registry credentials Option A — Prefect-managed registry credentials (recommended) - Create a DockerRegistryCredentials block (once):

python -c "
from prefect_docker.credentials import DockerRegistryCredentials
DockerRegistryCredentials(
    username='YOUR_DOCKERHUB_USERNAME',
    password='YOUR_DOCKERHUB_TOKEN_OR_PASSWORD',
    registry_url='<https://index.docker.io/v1/>'
).save('dockerhub-creds', overwrite=True)
"

- In the Prefect UI: - Go to Blocks → confirm your “Docker Registry Credentials” block exists (e.g., named

dockerhub-creds

) - Go to Work Pools → Docker → Edit default job variables - Set your default image (e.g., `docker.io/yourorg/yourimage:tag`) - Set “Registry credentials” to the

dockerhub-creds

block - Optionally set

image_pull_policy

to

Always

This centralizes the auth so any worker in the “Docker” pool can pull your private image. Option B — Host Docker login (per-VM) - On each VM, log into Docker Hub for the Docker daemon:

docker login -u YOUR_DOCKERHUB_USERNAME
# enter password/token when prompted

- Ensure your deployments (or pool defaults) specify the private image (e.g., `docker.io/yourorg/yourimage:tag`) - The worker will use the local Docker daemon’s credentials (~/.docker/config.json) 3) Install and start a worker on each VM - Install Prefect + the Docker worker plugin:

pip install -U "prefect" "prefect-docker"
# or: pip install -U "prefect[docker]"

- Authenticate the VM to your Prefect API (Cloud or Server): - Prefect Cloud: set

PREFECT_API_URL

and

PREFECT_API_KEY

- Prefect Server: set

PREFECT_API_URL

- Start the worker pointing to your pool:

prefect worker start --pool "Docker"

Repeat the “start” step on each VM — each will poll the “Docker” work pool and run containers locally on that host’s Docker daemon. 4) Deploy flows that use your Docker image - Set the deployment’s

image

via default job variables on the work pool (recommended) or in the deployment’s

job_variables

. - Build and push your image to Docker Hub (private) before running the deployment. Notes and tips - Docker Hub registry URL for credentials:

<https://index.docker.io/v1/>

- If your images are under

<http://docker.io/yourorg/|docker.io/yourorg/>...

, set

image_pull_policy: Always

to ensure workers pull updates - You can run

prefect worker start

under a process manager (systemd, supervisord, Docker) for auto-restart - Ensure each VM’s Docker can reach Docker Hub and has any required network/volume settings you add to job variables Helpful references - Work pools and workers overview - Blocks in Prefect (secure credentials) Questions to tailor the setup - Do you prefer managing Docker Hub auth centrally in Prefect (Block) or per-host via

docker login

? - Are you using Prefect Cloud or self-hosted Prefect Server? I can share the exact environment variables/steps for your setup.