<@ULVA73B9P> We have a docker work pool on a linux...
# ask-marvin
e
@Marvin We have a docker work pool on a linux VM, and we are running our prefect code as a docker container in this. A key part of the prefect code involves starting another container (container 2) which uses a different image to pull source data and load it into our snowflake destination. Where is this container 2 spun up? Inside the prefect docker worker? Outside? Inside the prefect code container? Can you give an overview of the architecture
m
thought for 119.3 seconds
Yes — in the setup you described, container 2 is usually spun up on the same Linux VM’s Docker daemon, not “inside” the Prefect worker container/process in a nested way. Short version: - The Prefect Docker worker talks to the VM’s Docker daemon - The worker starts your flow run container (
container 1
) - If code inside
container 1
starts another Docker container (
container 2
), that only works if
container 1
has access to the host Docker daemon, typically via
/var/run/docker.sock
- In that case,
container 2
is not nested inside
container 1
— it is a sibling container on the same host ## Architecture overview Typical layout:
Copy code
text
Linux VM
├── Docker daemon
├── Prefect Docker worker
│   └── asks Docker daemon to start flow run containers
├── Flow run container (container 1)
│   └── runs your Prefect flow code
└── Secondary container (container 2)
    └── started by your flow code, if it can access the Docker daemon
## What runs where ### 1. Prefect Docker worker The Docker worker typically runs on the VM and communicates with the host Docker daemon. Its job is to: - poll Prefect for work - create the container for the flow run - start it - monitor logs / exit status ### 2. Your Prefect code container (
container 1
) This is the container the worker launches for a flow run. Your flow code executes inside this container. ### 3. The second container (
container 2
) This depends on how your flow starts it. #### If your flow uses the host Docker socket If
container 1
has
/var/run/docker.sock
mounted, and your code uses Docker CLI or Docker SDK, then it is talking to the host Docker daemon. That means
container 2
is created by the host daemon and runs as a separate sibling container on the VM. So: - not inside the worker - not truly inside
container 1
- outside both, as another host-level container managed by Docker ## Important distinction: sibling, not nested People often think:
Copy code
text
container 1
└── container 2
But with Docker socket mounting, it’s really:
Copy code
text
VM Docker daemon
├── container 1
└── container 2
container 1
is just issuing Docker API calls to the host. ## When would
container 2
not start? By default, a flow container does not automatically have Docker access. So if your flow code tries to run Docker commands, it will only work if: - Docker CLI is installed in
container 1
, or Python Docker SDK is installed - and
/var/run/docker.sock
is mounted into
container 1
Without that,
container 1
usually cannot start other containers. ## Networking implications If
container 2
is launched through the host Docker daemon: - it gets its own container lifecycle - it gets its own network namespace - it is not automatically “inside” the network namespace of
container 1
So communication between them depends on: - Docker network configuration - container names - published ports - custom Docker network attachment ## Security note Mounting
/var/run/docker.sock
into your flow container is powerful but risky. It effectively gives code in
container 1
broad control over the host Docker daemon, including the ability to: - start/stop other containers - inspect host-managed containers - mount volumes - potentially escalate privileges on the host So this pattern is common, but should be used only with trusted code. ## Mental model A good mental model is: - Worker = orchestration agent - Flow container = where your Prefect code runs - Docker socket = “remote control” to the host Docker engine - Second container = another host container created through that remote control ## If helpful, I can also draw: - a more detailed architecture diagram - the exact Docker socket mount setup for a Prefect Docker work pool - or explain the difference between this and Docker-in-Docker
e
How can I view actively running prefect docker workers in linux?
@Marvin How can I view actively running prefect docker workers in linux?
m
thought for 21.9 seconds
If by “actively running Prefect docker workers” you mean the worker processes on your Linux machine, the answer depends on how you started them. ## 1. If the worker is running as a Linux process You can check processes like this:
Copy code
bash
ps aux | grep "prefect worker start"
or:
Copy code
bash
pgrep -af "prefect worker start"
That will show worker commands such as:
Copy code
bash
prefect worker start --pool my-docker-pool --type docker
## 2. If the worker itself is running in a Docker container Then list Docker containers on the host:
Copy code
bash
docker ps
You can filter for likely worker containers:
Copy code
bash
docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}\t{{.Status}}"
and inspect logs with:
Copy code
bash
docker logs <container_name_or_id>
If you know the worker container name:
Copy code
bash
docker exec -it <container_name_or_id> ps aux
## 3. If it is managed by systemd A lot of Linux VMs run workers as a service. Check:
Copy code
bash
systemctl list-units --type=service | grep prefect
and then:
Copy code
bash
systemctl status <service-name>
## 4. From Prefect’s side: check the work pool You can inspect the work pool itself:
Copy code
bash
prefect work-pool ls
and:
Copy code
bash
prefect work-pool inspect <pool-name>
However, work pools do not directly show you a live Linux process list of workers. They show the pool configuration in Prefect. ## Important nuance A Docker work pool does not mean the worker must be running in Docker. Usually: - worker process runs on the Linux VM - worker launches flow run containers via Docker So the worker is often visible with
ps
, while the flow runs are visible with
docker ps
. ## Most useful commands Try these first:
Copy code
bash
pgrep -af "prefect worker start"
docker ps
systemctl list-units --type=service | grep prefect
## CLI refs - Prefect docs -
prefect worker --help
-
prefect work-pool --help
If you want, I can also give you: - a one-liner to identify whether your worker is process-based vs container-based - or how to list currently running flow containers for a Docker work pool