Anton L.
10/26/2025, 8:34 PMMarvin
10/26/2025, 8:35 PMMarvin
10/26/2025, 8:42 PMprivileged: true
- /dev/shm size: pass shm_size through container_create_kwargs (this maps directly to Docker’s container create kwargs)
CLI steps
1) Fetch the default template:
prefect work-pool get-default-base-job-template --type docker > docker_template.json
2) Edit docker_template.json and set these under the "job_configuration" object:
{
"job_configuration": {
"privileged": true,
"container_create_kwargs": {
"shm_size": "2g"
}
}
}
Note: Keep the rest of the existing fields; just add/modify these.
3) Update the pool:
prefect work-pool update "my-docker-pool" --base-job-template docker_template.json
4) Verify:
prefect work-pool inspect "my-docker-pool"
UI steps
- Prefect UI -> Work Pools -> select your Docker pool -> Edit Base job template
- Under "job_configuration", add:
"privileged": true,
"container_create_kwargs": { "shm_size": "2g" }
- Save. New flow runs submitted to this pool will use these settings.
Notes
- shm_size can be a string like "2g" or an integer (bytes). Default is 64MB if unset.
- Changes apply to future jobs; running jobs are unaffected. No worker restart is required.
Kubernetes work pool
There’s no single “privileged” toggle; set container securityContext and mount a larger /dev/shm via an emptyDir volume.
CLI steps
1) Fetch the default template:
prefect work-pool get-default-base-job-template --type kubernetes > k8s_template.json
2) In k8s_template.json, update the container spec and volumes. For example, inside:
job_configuration.job_manifest.spec.template.spec.containers[0] add:
"securityContext": {
"privileged": true
},
"volumeMounts": [
{ "name": "dshm", "mountPath": "/dev/shm" }
]
And at job_configuration.job_manifest.spec.template.spec add:
"volumes": [
{
"name": "dshm",
"emptyDir": {
"medium": "Memory",
"sizeLimit": "2Gi"
}
}
]
3) Update the pool:
prefect work-pool update "my-k8s-pool" --base-job-template k8s_template.json
UI steps
- Prefect UI -> Work Pools -> select your Kubernetes pool -> Edit Base job template
- Add the container securityContext, volumeMounts, and top-level volumes as above, then save.
Notes
- Privileged containers require your cluster’s policies to allow it (Pod Security Admission or other controls).
- The emptyDir at /dev/shm gives you an in-memory shared memory segment sized by sizeLimit.
Per-deployment override (optional)
If you need this for only some flows, you can override at deployment time:
flow.from_source("<https://github.com/org/repo>", entrypoint="flows/my_flow.py:flow").deploy(
name="gpu-work",
work_pool_name="my-docker-pool",
job_variables={
"privileged": True,
"container_create_kwargs": {"shm_size": "2g"}
}
)
Docs
- Managing work pools: Manage work pools
- Docker worker: Docker work pools
- Kubernetes worker: Kubernetes work pools
Would you like me to tailor the exact JSON edits for your current pool type and template? If so, which worker type (Docker or Kubernetes) and what pool name are you using?