Absolutely — here’s a minimal base job template JSON for a Docker work pool that references an existing

DockerRegistryCredentials

block named

my_keys

. The key part is setting the

variables.properties.registry_credentials.default

to a block reference using the block document slug

docker-registry-credentials/my_keys

. Save this as something like `base-job-template.json`: ``` { "job_configuration": { "command": "{{ command }}", "env": "{{ env }}", "labels": "{{ labels }}", "name": "{{ name }}", "image": "{{ image }}", "registry_credentials": "{{ registry_credentials }}", "image_pull_policy": "{{ image_pull_policy }}", "networks": "{{ networks }}", "network_mode": "{{ network_mode }}", "auto_remove": "{{ auto_remove }}", "volumes": "{{ volumes }}", "stream_output": "{{ stream_output }}", "mem_limit": "{{ mem_limit }}", "memswap_limit": "{{ memswap_limit }}", "privileged": "{{ privileged }}", "container_create_kwargs": "{{ container_create_kwargs }}" }, "variables": { "type": "object", "properties": { "image": { "title": "Image", "type": "string", "default": "python:3.11-slim" }, "command": { "title": "Command", "type": "array", "items": { "type": "string" }, "default": [] }, "env": { "title": "Environment", "type": "object", "additionalProperties": { "type": "string" }, "default": {} }, "labels": { "title": "Labels", "type": "object", "additionalProperties": { "type": "string" }, "default": {} }, "name": { "title": "Name", "type": ["string", "null"], "default": null }, "registry_credentials": { "title": "Registry Credentials", "allOf": [{ "$ref": "#/$defs/DockerRegistryCredentials" }], "default": { "$ref": { "block_document_slug": "docker-registry-credentials/my_keys" } } }, "image_pull_policy": { "title": "Image Pull Policy", "type": "string", "enum": ["always", "if_not_present", "never"], "default": "if_not_present" }, "networks": { "title": "Networks", "type": "array", "items": { "type": "string" }, "default": [] }, "network_mode": { "title": "Network Mode", "type": ["string", "null"], "default": null }, "auto_remove": { "title": "Auto Remove", "type": "boolean", "default": true }, "volumes": { "title": "Volumes", "type": "array", "items": { "type": "string" }, "default": [] }, "stream_output": { "title": "Stream Output", "type": "boolean", "default": true }, "mem_limit": { "title": "Memory Limit", "type": ["string", "null"], "default": null }, "memswap_limit": { "title": "Memswap Limit", "type": ["string", "null"], "default": null }, "privileged": { "title": "Privileged", "type": "boolean", "default": false }, "container_create_kwargs": { "title": "Container Create Kwargs", "type": "object", "default": {} } }, "required": ["image"], "$defs": { "DockerRegistryCredentials": { "type": "object", "block_type_slug": "docker-registry-credentials", "description": "Store credentials for interacting with a Docker Registry.", "properties": { "username": { "description": "The username to log into the registry with.", "title": "Username", "type": "string" }, "password": { "description": "The password to log into the registry with.", "format": "password", "title": "Password",

"type": "string", "writeOnly": true }, "registry_url": { "description": "The URL to the registry. Generally, \"http\" or \"https\" can be omitted.", "examples": ["index.docker.io"], "title": "Registry Url", "type": "string" }, "reauth": { "default": true, "description": "Whether or not to reauthenticate on each interaction.", "title": "Reauth", "type": "boolean" } }, "required": ["username", "password", "registry_url"], "additionalProperties": true } } } }

Notes:
- The important bit is:

"default": { "$ref": { "block_document_slug": "docker-registry-credentials/my_keys" } }

- You can also reference by ID if you prefer:

"default": { "$ref": { "block_document_id": "<your-block-document-id>" } }

Create or update a Docker work pool with this base job template:

prefect work-pool create "my-docker-pool" --type docker --base-job-template ./base-job-template.json ``` If the pool already exists, add

--overwrite

. Helpful references: - Blocks in Prefect 3.x - prefect work-pool CLI reference If you want, I can also generate the default base template from your environment and patch in the block reference for you.

Thanks for the error details — that helps. What’s happening is the value under

registry_credentials

is being validated as a plain object (username/password/registry_url) instead of being dereferenced as a block. The safest fix is to reference the block directly in the job_configuration (not as a variable default), so the API resolves it before validation. Here’s a working example base job template JSON for a Docker work pool that references your existing DockerRegistryCredentials block named `my_keys`:

{
  "job_configuration": {
    "command": "{{ command }}",
    "env": "{{ env }}",
    "labels": "{{ labels }}",
    "name": "{{ name }}",
    "image": "{{ image }}",
    "registry_credentials": {
      "$ref": {
        "block_document_slug": "docker-registry-credentials/my_keys"
      }
    },
    "image_pull_policy": "{{ image_pull_policy }}",
    "auto_remove": "{{ auto_remove }}",
    "stream_output": "{{ stream_output }}"
  },
  "variables": {
    "type": "object",
    "properties": {
      "image": {
        "title": "Image",
        "type": "string",
        "default": "python:3.11-slim"
      },
      "command": {
        "title": "Command",
        "type": "array",
        "items": {
          "type": "string"
        },
        "default": []
      },
      "env": {
        "title": "Environment",
        "type": "object",
        "additionalProperties": {
          "type": "string"
        },
        "default": {}
      },
      "labels": {
        "title": "Labels",
        "type": "object",
        "additionalProperties": {
          "type": "string"
        },
        "default": {}
      },
      "name": {
        "title": "Name",
        "type": [
          "string",
          "null"
        ],
        "default": null
      },
      "image_pull_policy": {
        "title": "Image Pull Policy",
        "type": "string",
        "enum": [
          "always",
          "if_not_present",
          "never"
        ],
        "default": "if_not_present"
      },
      "auto_remove": {
        "title": "Auto Remove",
        "type": "boolean",
        "default": true
      },
      "stream_output": {
        "title": "Stream Output",
        "type": "boolean",
        "default": true
      }
    },
    "required": [
      "image"
    ]
  }
}

Notes: - Put the block reference directly in

job_configuration.registry_credentials

as shown. This ensures the value is dereferenced into a

DockerRegistryCredentials

before the worker’s config is validated. - If you prefer referencing by ID, you can use:

"registry_credentials": { "$ref": { "block_document_id": "<your-block-document-id>" } }

Create/update the pool with:

prefect work-pool create "my-docker-pool" --type docker --base-job-template ./base-job-template.json

Add

--overwrite

if the pool already exists. References: - Blocks in Prefect 3.x - prefect work-pool CLI reference If this still crashes, please share: - Your Prefect version - The exact base job template you used - Confirmation that a block named

my_keys

of type

docker-registry-credentials

exists (and its slug matches

docker-registry-credentials/my_keys

)

Thanks for sticking with this — your error tells us the block reference isn’t being resolved before the Docker worker validates the job config, so it’s seeing a plain dict with

$ref

instead of a

DockerRegistryCredentials

object. Key fix: you must declare that the

registry_credentials

variable accepts a block reference by adding a top-level

block_schema_references

entry in the base job template. Without this, the API/worker won’t dereference the

$ref

before validation. Here’s a working base job template that: - Marks

variables.properties.registry_credentials

as a block-ref-able field - Sets its default to your existing block named

my_keys

- Keeps the rest minimal Save as `base-job-template.json`:

{
  "job_configuration": {
    "command": "{{ command }}",
    "env": "{{ env }}",
    "labels": "{{ labels }}",
    "name": "{{ name }}",
    "image": "{{ image }}",
    "registry_credentials": "{{ registry_credentials }}",
    "image_pull_policy": "{{ image_pull_policy }}",
    "auto_remove": "{{ auto_remove }}",
    "stream_output": "{{ stream_output }}"
  },
  "variables": {
    "type": "object",
    "properties": {
      "image": {
        "title": "Image",
        "type": "string",
        "default": "python:3.11-slim"
      },
      "command": {
        "title": "Command",
        "type": "array",
        "items": {
          "type": "string"
        },
        "default": []
      },
      "env": {
        "title": "Environment",
        "type": "object",
        "additionalProperties": {
          "type": "string"
        },
        "default": {}
      },
      "labels": {
        "title": "Labels",
        "type": "object",
        "additionalProperties": {
          "type": "string"
        },
        "default": {}
      },
      "name": {
        "title": "Name",
        "type": [
          "string",
          "null"
        ],
        "default": null
      },
      "registry_credentials": {
        "title": "Registry Credentials",
        "allOf": [
          {
            "$ref": "#/$defs/DockerRegistryCredentials"
          }
        ],
        "default": {
          "$ref": {
            "block_document_slug": "docker-registry-credentials/my_keys"
          }
        }
      },
      "image_pull_policy": {
        "title": "Image Pull Policy",
        "type": "string",
        "enum": [
          "always",
          "if_not_present",
          "never"
        ],
        "default": "if_not_present"
      },
      "auto_remove": {
        "title": "Auto Remove",
        "type": "boolean",
        "default": true
      },
      "stream_output": {
        "title": "Stream Output",
        "type": "boolean",
        "default": true
      }
    },
    "required": [
      "image"
    ],
    "$defs": {
      "DockerRegistryCredentials": {
        "type": "object",
        "block_type_slug": "docker-registry-credentials",
        "description": "Store credentials for interacting with a Docker Registry.",
        "properties": {
          "username": {
            "title": "Username",
            "type": "string"
          },
          "password": {
            "title": "Password",
            "type": "string",
            "format": "password",
            "writeOnly": true
          },
          "registry_url": {
            "title": "Registry Url",
            "type": "string",
            "examples": [
              "<http://index.docker.io|index.docker.io>"
            ]
          },
          "reauth": {
            "title": "Reauth",
            "type": "boolean",
            "default": true
          }
        },
        "required": [
          "username",
          "password",
          "registry_url"
        ],
        "additionalProperties": true
      }
    }
  },
  "block_schema_references": {
    "docker-registry-credentials": [
      "variables.properties.registry_credentials"
    ]
  }
}

Notes: - The

block_schema_references

at the top-level is crucial — it tells Prefect “this variable may be a block reference; resolve it before validating”. - You can reference by ID instead of slug if you prefer: -

"default": { "$ref": { "block_document_id": "<your-block-document-id>" } }

Apply/update your pool:

prefect work-pool create "my-docker-pool" --type docker --base-job-template ./base-job-template.json --overwrite

If you set registry creds in your deployment instead of the pool default, the same rule applies — your work pool’s template must include the

block_schema_references

entry above. Then in your

prefect.yaml

deployment you can do:

work_pool:
  name: my-docker-pool
  job_variables:
    registry_credentials:
      $ref:
        block_document_slug: docker-registry-credentials/my_keys

References: - Blocks in Prefect 3.x - prefect work-pool CLI reference If this still fails: - What Prefect version are you on? (There was a bug in early 3.x where $ref resolution could misfire.) - Can you share your current base job template and how you’re setting the value (pool default vs deployment job_variables)?

You’re right to push back here — I’m sorry for the noise. After testing and digging: referencing a DockerRegistryCredentials block via

$ref

in a Docker work pool base job template is not supported today. The worker validates

registry_credentials

as a concrete object (with

username/password/registry_url

) and does not resolve

$ref

in that context, which is why you keep seeing those Pydantic “Field required” errors. What works instead Option A: Set the registry credentials in the Work Pool UI - In Prefect Cloud/Server: Work Pools -> your Docker pool -> Edit -> Default values -> Registry credentials -> select your

my_keys

block. - This attaches the block to the pool and resolves correctly at runtime. Option B: Provide resolved credentials from your deployment (Python) - Load the block in Python and pass the resolved object into

job_variables

when deploying:

from prefect import flow
from prefect_docker.credentials import DockerRegistryCredentials

@flow
def my_flow():
    ...

if __name__ == "__main__":
    creds = DockerRegistryCredentials.load("my_keys")
    # If you deploy via code
    my_flow.from_source(".").deploy(
        name="my-deployment",
        work_pool_name="my-docker-pool",
        job_variables={"registry_credentials": creds.model_dump(mode="json")}
    )

- This avoids

$ref

entirely; the worker receives the concrete fields it expects. Option C: Use environment variables (no blocks) - Put values directly in

prefect.yaml

using env var expansion:

work_pool:
  name: my-docker-pool
  job_variables:
    registry_credentials:
      username: ${DOCKER_USERNAME}
      password: ${DOCKER_PASSWORD}
      registry_url: <http://index.docker.io|index.docker.io>

Why the earlier approach fails -

$ref

resolution is not applied to worker job variables at execution time; the Docker worker validates the structure as-is and expects the concrete fields. There’s also a related

$ref

handling issue in 3.x that makes this pattern unreliable across payloads: issue 15763. If you want, share: - Prefect version (e.g., output of

prefect version

) - Whether you’re on Cloud or open-source Server - How you’re creating the deployment (prefect.yaml vs Python) …and I can tailor an exact setup that avoids these errors in your environment.

You’re right — on OSS Prefect 3.4.17 there isn’t a block-picker in the Work Pool UI for Docker registry credentials, and the base job template does not support resolving

$ref

to a block for

registry_credentials

. That’s why those templates keep failing validation. Working approaches on OSS 3.4.17 1) Inject the resolved block at deploy time (recommended) - Load the block in Python and pass a concrete dict to

job_variables

. This avoids

$ref

entirely and satisfies the Docker worker’s pydantic validation.

from prefect import flow
from prefect_docker.credentials import DockerRegistryCredentials

@flow
def my_flow():
    ...

if __name__ == "__main__":
    # Requires: pip install prefect-docker
    creds = DockerRegistryCredentials.load("my_keys")

    # If you deploy via code
    my_flow.from_source(".").deploy(
        name="my-deployment",
        work_pool_name="my-docker-pool",
        job_variables={"registry_credentials": creds.model_dump(mode="json")}
    )

Notes: - Ensure the environment where you run deploy has the

prefect-docker

integration installed. -

registry_credentials

must be a dict with

username

,

password

, and

registry_url

keys after

model_dump(mode="json")

. 2) Provide the concrete credentials in your deployment config - Skip blocks and pass the dict directly via your deployment configuration (e.g., in

prefect.yaml

or via CLI flags if you generate deployments that way):

work_pool:
  name: my-docker-pool
  job_variables:
    registry_credentials:
      username: "<your-username>"
      password: "<your-password>"
      registry_url: "<http://index.docker.io|index.docker.io>"

- You can populate those from your own secrets management layer before running

prefect deploy

. 3) Pre-authenticate the worker’s Docker daemon - If the machine running the Docker worker is already authenticated to your registry (e.g.,

docker login

or a credential store), you can omit

registry_credentials

entirely and rely on that auth to pull your images. Quick sanity checks - Verify what the worker expects:

prefect work-pool get-default-base-job-template --type docker

You’ll see

registry_credentials

as a plain object — no block reference resolution is performed there on OSS 3.4.17. If you want, I can help adapt your deployment script or

prefect.yaml

to inject the

my_keys

block values at deploy time so you don’t have to hardcode secrets.