Ed Hartley
07/01/2026, 9:09 AMMarvin
07/01/2026, 9:10 AMMarvin
07/01/2026, 9:12 AMcontainer 1)
- If code inside container 1 starts another Docker container (container 2), that only works if container 1 has access to the host Docker daemon, typically via /var/run/docker.sock
- In that case, container 2 is not nested inside container 1 — it is a sibling container on the same host
## Architecture overview
Typical layout:
text
Linux VM
├── Docker daemon
├── Prefect Docker worker
│ └── asks Docker daemon to start flow run containers
├── Flow run container (container 1)
│ └── runs your Prefect flow code
└── Secondary container (container 2)
└── started by your flow code, if it can access the Docker daemon
## What runs where
### 1. Prefect Docker worker
The Docker worker typically runs on the VM and communicates with the host Docker daemon.
Its job is to:
- poll Prefect for work
- create the container for the flow run
- start it
- monitor logs / exit status
### 2. Your Prefect code container (container 1)
This is the container the worker launches for a flow run.
Your flow code executes inside this container.
### 3. The second container (container 2)
This depends on how your flow starts it.
#### If your flow uses the host Docker socket
If container 1 has /var/run/docker.sock mounted, and your code uses Docker CLI or Docker SDK, then it is talking to the host Docker daemon.
That means container 2 is created by the host daemon and runs as a separate sibling container on the VM.
So:
- not inside the worker
- not truly inside container 1
- outside both, as another host-level container managed by Docker
## Important distinction: sibling, not nested
People often think:
text
container 1
└── container 2
But with Docker socket mounting, it’s really:
text
VM Docker daemon
├── container 1
└── container 2
container 1 is just issuing Docker API calls to the host.
## When would container 2 not start?
By default, a flow container does not automatically have Docker access.
So if your flow code tries to run Docker commands, it will only work if:
- Docker CLI is installed in container 1, or Python Docker SDK is installed
- and /var/run/docker.sock is mounted into container 1
Without that, container 1 usually cannot start other containers.
## Networking implications
If container 2 is launched through the host Docker daemon:
- it gets its own container lifecycle
- it gets its own network namespace
- it is not automatically “inside” the network namespace of container 1
So communication between them depends on:
- Docker network configuration
- container names
- published ports
- custom Docker network attachment
## Security note
Mounting /var/run/docker.sock into your flow container is powerful but risky.
It effectively gives code in container 1 broad control over the host Docker daemon, including the ability to:
- start/stop other containers
- inspect host-managed containers
- mount volumes
- potentially escalate privileges on the host
So this pattern is common, but should be used only with trusted code.
## Mental model
A good mental model is:
- Worker = orchestration agent
- Flow container = where your Prefect code runs
- Docker socket = “remote control” to the host Docker engine
- Second container = another host container created through that remote control
## If helpful, I can also draw:
- a more detailed architecture diagram
- the exact Docker socket mount setup for a Prefect Docker work pool
- or explain the difference between this and Docker-in-DockerEd Hartley
07/01/2026, 11:10 AMEd Hartley
07/01/2026, 11:10 AMMarvin
07/01/2026, 11:10 AMMarvin
07/01/2026, 11:11 AMbash
ps aux | grep "prefect worker start"
or:
bash
pgrep -af "prefect worker start"
That will show worker commands such as:
bash
prefect worker start --pool my-docker-pool --type docker
## 2. If the worker itself is running in a Docker container
Then list Docker containers on the host:
bash
docker ps
You can filter for likely worker containers:
bash
docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}\t{{.Status}}"
and inspect logs with:
bash
docker logs <container_name_or_id>
If you know the worker container name:
bash
docker exec -it <container_name_or_id> ps aux
## 3. If it is managed by systemd
A lot of Linux VMs run workers as a service. Check:
bash
systemctl list-units --type=service | grep prefect
and then:
bash
systemctl status <service-name>
## 4. From Prefect’s side: check the work pool
You can inspect the work pool itself:
bash
prefect work-pool ls
and:
bash
prefect work-pool inspect <pool-name>
However, work pools do not directly show you a live Linux process list of workers. They show the pool configuration in Prefect.
## Important nuance
A Docker work pool does not mean the worker must be running in Docker.
Usually:
- worker process runs on the Linux VM
- worker launches flow run containers via Docker
So the worker is often visible with ps, while the flow runs are visible with docker ps.
## Most useful commands
Try these first:
bash
pgrep -af "prefect worker start"
docker ps
systemctl list-units --type=service | grep prefect
## CLI refs
- Prefect docs
- prefect worker --help
- prefect work-pool --help
If you want, I can also give you:
- a one-liner to identify whether your worker is process-based vs container-based
- or how to list currently running flow containers for a Docker work pool