Mark
08/28/2025, 10:54 AMMarvin
08/28/2025, 10:54 AMMarvin
08/28/2025, 11:04 AMregistry_credentials
field.
Here’s a reliable way to do it in Prefect 3.x:
1) Save your DockerRegistryCredentials block (via code or UI)
from prefect_docker.credentials import DockerRegistryCredentials
DockerRegistryCredentials(
username="myuser",
password="mytoken", # e.g., a PAT for <http://ghcr.io|ghcr.io> or Docker Hub token
registry_url="<http://index.docker.io|index.docker.io>"
).save("my-registry", overwrite=True)
- Docs on blocks: Blocks in Prefect 3
2) Export the default Docker work pool base job template
prefect work-pool get-default-base-job-template --type docker --file base.json
3) Edit base.json to reference your saved block
In the job_configuration
section, set registry_credentials
to a block reference:
{
"job_configuration": {
"command": "{{ command }}",
"env": "{{ env }}",
"labels": "{{ labels }}",
"name": "{{ name }}",
"image": "{{ image }}",
"registry_credentials": "{{ block('docker-registry-credentials/my-registry') }}",
"image_pull_policy": "{{ image_pull_policy }}",
"networks": "{{ networks }}",
"network_mode": "{{ network_mode }}",
"auto_remove": "{{ auto_remove }}",
"volumes": "{{ volumes }}",
"stream_output": "{{ stream_output }}",
"mem_limit": "{{ mem_limit }}",
"memswap_limit": "{{ memswap_limit }}",
"privileged": "{{ privileged }}",
"container_create_kwargs": "{{ container_create_kwargs }}"
},
"variables": {
"...": "keep the rest as-is"
}
}
That tells any Docker worker using this pool to authenticate with your saved block when pulling private images.
4) Create or update the work pool with this template
- Create:
prefect work-pool create my-docker-pool --type docker --base-job-template base.json
- Update:
prefect work-pool update my-docker-pool --base-job-template base.json
Optional: override per-deployment instead of at the pool
- In prefect.yaml:
deployments:
- name: prod
work_pool: my-docker-pool
image: "<http://index.docker.io/myuser/myapp:{{|index.docker.io/myuser/myapp:{{> git_commit }}"
job_variables:
registry_credentials: "{{ block('docker-registry-credentials/my-registry') }}"
- Or in Python:
from prefect_docker.credentials import DockerRegistryCredentials
creds = DockerRegistryCredentials.load("my-registry")
flow.from_source("<https://github.com/org/repo>").deploy(
name="prod",
work_pool_name="my-docker-pool",
image="<http://index.docker.io/myuser/myapp:latest|index.docker.io/myuser/myapp:latest>",
job_variables={"registry_credentials": creds},
)
Notes:
- If your worker host already has docker login
configured (e.g., via ~/.docker/config.json), you can skip passing registry credentials via Prefect.
- Ensure your worker environment has the Docker integration installed: pip install 'prefect[docker]'
.
If you want, share your registry (Docker Hub, ghcr.io, ECR, etc.) and how you’re deploying (prefect.yaml vs. Python), and I can tailor the exact snippet.