<@ULVA73B9P> I deployed a single Cloud Run v2 work...
# marvin-ai
m
@Marvin I deployed a single Cloud Run v2 worker to my work pool, but the prefect server shows two different workers, each polling within about a second of each other. I have verified there is only one Cloud Run v2 Worker. Any ideas why the prefect server shows two workers when there is only one?
m
thought for 510.1 seconds
Short answer: if the UI shows two workers “polling” within a second of each other, it almost always means two worker processes are actually connected to that work pool. With Cloud Run v2, this can happen if Cloud Run briefly or continuously runs more than one instance/revision of your worker container, or if you’ve accidentally started a worker elsewhere (e.g., your laptop) pointing at the same pool. Common causes to check - Two Cloud Run instances or revisions - Check Cloud Run > Revisions and ensure only one revision has traffic. - Check Metrics for “Container instance count” to confirm a single instance. - Temporarily set max instances = 1 (and optionally min instances = 1) to rule out autoscaling starting a second worker. - Worker restarts creating a second process - If you don’t pass a
--name
, Prefect generates a new random name on each start. If Cloud Run restarts the container, you’ll see a new worker entry. Note: old entries can linger as “connected” for several minutes, but they won’t show new polls unless the old process is still alive. Seeing both “polling” means both are actively running. - A second worker running elsewhere - Double-check no local/CI/another machine is running
prefect worker start -p <your-pool>
. - Container starts multiple worker processes - Ensure your container CMD/entrypoint starts exactly one worker process (no wrapper that spawns multiple processes). How to verify quickly - In the Prefect UI, open each worker’s detail panel and compare: - Worker name, “Started at”, Prefect version/OS, and “Last polled”. - If names look randomly generated, you likely didn’t set
--name
and had at least two starts. - In Cloud Run Logs, search for the “Starting worker” log lines to see if two containers started around the same time. - In Cloud Run settings, confirm: - Max instances = 1 (for debugging). - Only one revision has traffic. - Concurrency set appropriately. - If CPU is not always allocated, Cloud Run can suspend/resume the container which may trigger restarts. Recommendations - Give the worker a fixed name so it’s easy to identify restarts/duplicates:
Copy code
prefect worker start -p <your-pool> --name cloud-run-v2-worker
- While debugging, cap Cloud Run to a single instance: - Set max instances = 1 and confirm only one revision with traffic. - Ensure your container entrypoint only runs one worker command and doesn’t fork extra processes. FYI: worker health checks/heartbeats can keep stale entries around for a bit after a restart, but those won’t show new polls. Seeing two workers both polling almost always indicates two active processes. More on worker health/heartbeats: Worker health checks in Prefect 3.x and general background on workers: Work pools and workers overview. If you can share: - The two worker names shown in the UI - How you deployed the Cloud Run service (command/entrypoint and settings for min/max instances and revisions) - Whether you’ve ever run
prefect worker start
locally against the same pool …I can help pinpoint the exact cause.