Nils
03/19/2026, 12:23 PM"container_create_kwargs": {
"extra_hosts": {
"a": "ip"
}
}
Prefect version 3.6.13Marvin
03/19/2026, 12:24 PMMarvin
03/19/2026, 12:27 PMextra_hosts should be passed via container_create_kwargs in the job variables (deployment) or in the work pool’s base job template. If it’s “staying empty”, it’s usually one of these:
- The setting is in the wrong place (not under work_pool.job_variables.container_create_kwargs)
- A base job template is overriding it
- The worker is not picking up the latest deployment/job variables
- The value is being dropped during merge because it’s not in the expected shape
A quick checklist and a working example:
1) Put it under job variables for your deployment (correct nesting)
deployments:
- name: my-deployment
entrypoint: flow.py:my_flow
work_pool:
name: my-docker-pool
job_variables:
container_create_kwargs:
extra_hosts:
a: "1.2.3.4"
Or programmatically:
flow.from_source(...).deploy(
name="my-deployment",
work_pool_name="my-docker-pool",
job_variables={
"container_create_kwargs": {
"extra_hosts": {
"a": "1.2.3.4"
}
}
}
)
2) Valid shapes for extra_hosts
- Dict (recommended): {"a": "1.2.3.4"}
- Or list of strings: ["a:1.2.3.4"]
Your dict form is correct; just be sure the IP is a valid IP address.
3) Verify what the deployment actually has
Run:
prefect deployment inspect "<flow-name>/<deployment-name>"
and check the work_pool.job_variables.container_create_kwargs.extra_hosts section.
4) Verify in the worker/container
- In the Docker worker logs (set worker log level to DEBUG if possible), look for the final resolved container_create_kwargs.
- On a run container, check:
docker inspect <container_id> | jq '.[0].HostConfig.ExtraHosts'
You should see something like:
["a:1.2.3.4", "host.docker.internal:host-gateway"] # on Linux
Notes:
- On Linux, the Docker worker will also add host.docker.internal:host-gateway automatically; your entries should merge with that (not overwrite).
- Make sure you’re using a Docker work pool/worker (not Kubernetes), and that the worker process is running with a version of the prefect-docker integration that matches your Prefect version (e.g., pip show prefect-docker in the worker environment).
If it’s still empty, could you share:
- The output of prefect deployment inspect "<flow>/<deployment>" (just the work_pool.job_variables portion)
- The Docker worker logs around container creation
- The result of pip show prefect-docker from the worker environment
That’ll help pinpoint whether it’s a placement/merge issue or something environmental.Nils
03/19/2026, 12:31 PMMarvin
03/19/2026, 12:31 PMMarvin
03/19/2026, 12:33 PMprefect deployment inspect "<flow-name>/<deployment-name>"
Check work_pool.job_variables.container_create_kwargs.extra_hosts.
- Work pool:
prefect work-pool inspect "<your-work-pool>"
Check the base job template for container_create_kwargs.extra_hosts.
2) Watch the worker pick up changes
- Check the worker logs after you update the deployment/job template. You should see a heartbeat/sync within ~30s. The next submitted run will use the updated template/variables.
- Run another flow and then inspect the created container:
docker inspect <container_id> | jq '.[0].HostConfig.ExtraHosts'
You should see your entries (and possibly host.docker.internal:host-gateway on Linux).
3) Force a quick confirmation
- After updating the deployment/job variables, immediately trigger a new run. If the container still lacks your hosts, it means the worker used older config. Wait ~30s after changing the work pool template (for heartbeat), then trigger another run and re-check.
If it still looks stale
- Ensure you edited the right place (deployment job_variables vs work pool template) and the nesting is correct under work_pool.job_variables.container_create_kwargs.extra_hosts.
- Make sure the worker is connected to the same workspace and work pool you updated.
- Consider enabling more verbose logs on the worker to see the resolved configuration just before container creation.
If you share:
- The output of prefect deployment inspect "<flow>/<deployment>" (job_variables portion)
- prefect work-pool inspect "<pool>"
- A snippet of the worker logs around a submission
I can help pinpoint exactly where it’s getting out of sync.