<@ULVA73B9P> ```Worker 'CloudRunWorkerV2 20b3739b-...
# ask-marvin
j
@Marvin
Copy code
Worker 'CloudRunWorkerV2 20b3739b-deb6-46d5-9ea5-479b01fc7516' submitting flow run '6a24cec0-11d2-417a-9531-ce9c32aedc38'
09:24:07 AM
prefect.flow_runs.worker
PREFECT_API_AUTH_STRING is provided as a plaintext environment variable. For better security, consider providing it as a secret using 'prefect_api_auth_string_secret' or 'env_from_secrets' in your base job template.
09:24:07 AM
prefect.flow_runs.worker
Creating Cloud Run JobV2 berserk-baboon-793ca844a1b348d98c2013331a8710a4
09:24:07 AM
prefect.flow_runs.worker
Current Job Condition: waiting for condition update
09:24:08 AM
prefect.flow_runs.worker
Current Job Condition: {'type': 'Ready', 'state': 'CONDITION_RECONCILING', 'lastTransitionTime': '2025-11-18T17:24:08.227442Z'}
09:24:13 AM
prefect.flow_runs.worker
Submitting Cloud Run Job V2 berserk-baboon-793ca844a1b348d98c2013331a8710a4 for execution...
09:24:18 AM
prefect.flow_runs.worker
Cloud Run Job V2 berserk-baboon-793ca844a1b348d98c2013331a8710a4 submitted for execution with command: u v   r u n   - - n o - d e v   p r e f e c t   f l o w - r u n   e x e c u t e
09:24:18 AM
prefect.flow_runs.worker
Completed submission of flow run '6a24cec0-11d2-417a-9531-ce9c32aedc38'
I have a cloudrun gcp setup as a worker pool but the run is hanging in pending state. I don't see anything in the worker logs except that the job was submitted. What else can I check to see what's wrong?
m
thought for 596.3 seconds
Thanks for the details — “Pending” in Prefect usually means the Cloud Run job either never started or failed before the Prefect engine could begin. Since your worker shows “Job submitted” but the flow stays Pending, the next step is to look at the Cloud Run Job execution and its logs. Here’s a focused checklist to find the issue quickly: 1) Verify the Cloud Run Job and its Execution - In the GCP console: Cloud Run -> Jobs -> locate the job name from your worker log (
berserk-baboon-...
) -> Executions. - Or via gcloud:
Copy code
gcloud run jobs describe berserk-baboon-793ca844a1b348d98c2013331a8710a4 --region=<your-region>
gcloud run jobs executions list --job=berserk-baboon-793ca844a1b348d98c2013331a8710a4 --region=<your-region>
gcloud run jobs executions describe <execution-name> --region=<your-region>
- If there is no execution listed, the worker may lack permission to start executions (see IAM below). 2) Pull the Cloud Run execution logs - In Logs Explorer, run a query for the job:
Copy code
resource.type="cloud_run_job"
resource.labels.job_name="berserk-baboon-793ca844a1b348d98c2013331a8710a4"
- Or via gcloud:
Copy code
gcloud logging read 'resource.type=cloud_run_job AND resource.labels.job_name="berserk-baboon-793ca844a1b348d98c2013331a8710a4"' --limit=200 --format=json
You’re looking for container startup errors like “Failed to pull image”, “uv: command not found”, missing env vars, Secret Manager access errors, VPC connector errors, etc. 3) Confirm the image and command are valid - Prefect 3 Cloud Run worker uses the official image and uv by default: - Image:
prefecthq/prefect:3-latest
- Command shown spaced-out in logs is actually:
uv run --no-dev prefect flow-run execute
- If you use a custom image, ensure it contains
uv
(or switch to the official Prefect 3 image). If
uv
is missing, the container will crash immediately and you’ll only see that in Cloud Run logs. 4) Check env vars made it into the job - The container must have
PREFECT_API_URL
and
PREFECT_API_KEY
(or
PREFECT_API_AUTH_STRING
). - Inspect the job’s container env:
Copy code
gcloud run jobs describe berserk-baboon-793ca844a1b348d98c2013331a8710a4 --region=<your-region> --format=json | jq '.spec.template.template.containers[0].env'
- If you’re using
env_from_secrets
or
prefect_api_key_secret
, the job’s service account needs Secret Manager access (see IAM). 5) IAM: two service accounts often involved - The identity that creates/submits the Job (your worker’s credentials) needs: - roles/run.admin - roles/iam.serviceAccountUser (to use the job’s execution SA if it’s different) - The Job’s execution service account needs whatever the flow requires, commonly: - Secret Manager access: roles/secretmanager.secretAccessor (if using
env_from_secrets
or
prefect_api_key_secret
) - VPC/connector permissions if you set a VPC connector - If no execution is created, it’s often a permission mismatch (run.admin or serviceAccountUser). 6) Networking/VPC connector - If you set a
vpc_connector_name
but it’s misconfigured, startup can fail before logs reach Prefect. - Validate connector exists and is Ready:
Copy code
gcloud compute networks vpc-access connectors describe <connector-name> --region=<your-region>
7) Optional: keep the job for inspection - In your work pool variables, set
keep_job: true
temporarily so the job remains and you can re-describe/inspect it after a failure. 8) Turn up worker logs (optional) - If you’re running a local/hosted worker, start it with debug to see the exact job names and statuses:
Copy code
PREFECT_LOGGING_LEVEL=DEBUG prefect worker start --pool <your-pool-name>
Common causes you’ll likely see in Cloud Run logs - Custom image missing
uv
→ Use
prefecthq/prefect:3-latest
or add uv. - Missing Prefect env vars → Ensure
PREFECT_API_URL
+
PREFECT_API_KEY
are present. - Secret Manager errors → Add roles/secretmanager.secretAccessor to the job’s service account.
- PermissionDenied creating execution → Grant roles/run.admin + roles/iam.serviceAccountUser to the worker identity. - Image pull failures → Check the image name/tag and registry access. Docs and references - GCP worker guide (Cloud Run): Prefect GCP worker guide - Serverless work pools overview: Serverless work pools (push) - Troubleshooting cloud runs: Troubleshooting guide If you can share: - Whether you’re using the official Prefect image or a custom image - The execution status/error from Cloud Run (from step 2) - Whether you’re using
env_from_secrets
or
prefect_api_key_secret
I can pinpoint the fix quickly.