Jussi Kämäräinen
10/06/2025, 11:47 AMcloud-run:push workpool to cloud-run-v2:push workpool. The flow run executions start as intended but they give warning:
Unable to check status of flow run: <HttpError 404 when requesting <https://run.googleapis.com/v2/projects/project-name/locations/europe-north1/jobs/prefect-dbt-run-24h-2025-10-06T09:50:09/executions/prefect-wakeful-dragon-2zmln?alt=json> returned "Resource 'prefect-wakeful-dragon-2zmln' of kind 'EXECUTION' in region 'europe-north1' in project 'project-name' with parent 'prefect-dbt-run-24h-2025-10-06T09:50:09' does not exist.". Details: "Resource 'prefect-wakeful-dragon-2zmln' of kind 'EXECUTION' in region 'europe-north1' in project 'project-name' with parent 'prefect-dbt-run-24h-2025-10-06T09:50:09' does not exist.">
I have confirmed the related job prefect-wakeful-dragon-2zmln exists in Cloud Run job pool and it has a YAML spec starting with:
apiVersion: <http://run.googleapis.com/v1|run.googleapis.com/v1>
kind: Execution
metadata:
name: prefect-wakeful-dragon-2zmln
namespace: '837612869087'
selfLink: /apis/run.googleapis.com/v1/namespaces/837612869087/executions/prefect-wakeful-dragon-2zmln
uid: 2a904416-a9d6-4e6c-a17a-638c75c777f8
resourceVersion: AAZAen54gj0
generation: 1
creationTimestamp: '2025-10-06T09:49:16.645383Z'
It seems like the workpool is still generating v1 jobs even though I have set it to be v2. I have confirmed from Prefect UI that the workpool is indeed of type Cloud Run V2:push. What could be the issue?Marvin
10/06/2025, 11:47 AMMarvin
10/06/2025, 11:53 AM{{ flow_run.id }}
or, if you prefer something readable:
{{ deployment.name | lower }}-{{ flow_run.id }}
If you really want to keep a timestamp, make it safe:
{{ deployment.name | lower }}-{{ scheduled_start_time | replace(':','-') | replace('T','-') | lower }}
2) Override the job name in the Cloud Run V2 work pool’s base job template
- Open your Cloud Run V2:push work pool → Base job template.
- Set the job name field to a DNS-safe template, e.g.:
{{ flow_run.id }}
or
{{ deployment.name | lower }}-{{ flow_run.id }}
- Save, then re-run.
Also recommended
- Update to the latest versions of prefect and prefect-gcp to pick up any recent fixes or improvements.
- Our GCP worker guide is here: Prefect GCP worker guide
If you’re still seeing the warning after switching to a DNS-safe name, please share:
- Your prefect and prefect-gcp versions
- The deployment run name template
- The Cloud Run V2 work pool base job template (redact secrets)
- A short snippet from the worker logs showing the “Creating Cloud Run job/execution” and the names it uses
I can help pinpoint whether there’s a naming mismatch in polling or something else we need to tweak.Jussi Kämäräinen
10/06/2025, 7:36 PMf"dbt-run-24h-{datetime.now().strftime("%Y-%m-%d-%H-%M-%S")}" so I removed the problematic characters. I'm still getting the same warnings on flow runs.
I'm using:
prefect==3.4.12
prefect-gcp==0.6.10
Here is the base job template:
{
"variables": {
"type": "object",
"required": [
"credentials"
],
"properties": {
"cpu": {
"type": "string",
"title": "CPU",
"default": "1000m",
"pattern": "^(\\d*000)m$",
"description": "The CPU to allocate to the Cloud Run job."
},
"env": {
"type": "object",
"title": "Environment Variables",
"default": {
"prefect_var_env": "test",
"prefect_var_region": "europe-north1",
"prefect_var_gcp_account_id": "project-name"
},
"description": "Environment variables to set when starting a flow run.",
"additionalProperties": {
"type": "string"
}
},
"args": {
"type": "array",
"items": {
"type": "string"
},
"title": "Args",
"description": "The arguments to pass to the Cloud Run Job V2's entrypoint command."
},
"name": {
"type": "string",
"title": "Name",
"description": "Name given to created infrastructure."
},
"image": {
"type": "string",
"title": "Image Name",
"default": "prefecthq/prefect:3.4.12-python3.12",
"description": "The image to use for the Cloud Run job. If not provided the latest Prefect image will be used."
},
"labels": {
"type": "object",
"title": "Labels",
"description": "Labels applied to created infrastructure.",
"additionalProperties": {
"type": "string"
}
},
"memory": {
"type": "string",
"title": "Memory",
"default": "512Mi",
"example": "512Mi",
"pattern": "^\\d+(?:G|Gi|M|Mi)$",
"description": "The memory to allocate to the Cloud Run job along with the units, which could be: G, Gi, M, Mi."
},
"region": {
"type": "string",
"title": "Region",
"default": "europe-north1",
"example": "europe-north1",
"description": "The region in which to run the Cloud Run job"
},
"command": {
"type": "string",
"title": "Command",
"description": "The command to use when starting a flow run. In most cases, this should be left blank and the command will be automatically generated."
},
"timeout": {
"type": "integer",
"title": "Job Timeout",
"default": 3600,
"maximum": 86400,
"description": "The length of time that Prefect will wait for a Cloud Run Job to complete before raising an exception (maximum of 86400 seconds, 1 day).",
"exclusiveMinimum": 0
},
"keep_job": {
"type": "boolean",
"title": "Keep Job After Completion",
"default": false,
"description": "Keep the completed Cloud run job on Google Cloud Platform."
},
"credentials": {
"anyOf": [
{
"$ref": "#/definitions/GcpCredentials"
},
{
"$ref": "#/definitions/OidcGcpCredentials"
}
],
"title": "GCP Credentials",
"default": {
"$ref": {
"block_document_id": "redacted-uuid-reference-id"
}
},
"description": "The GCP Credentials used to initiate the Cloud Run Job. `service_account_info` is required."
},
"max_retries": {
"type": "integer",
"title": "Max Retries",
"default": 0,
"description": "The number of times to retry the Cloud Run job."
},
"launch_stage": {
"enum": [
"ALPHA",
"BETA",
"GA",
"DEPRECATED",
"EARLY_ACCESS",
"PRELAUNCH",
"UNIMPLEMENTED",
"LAUNCH_TAG_UNSPECIFIED"
],
"type": "string",
"title": "Launch Stage",
"default": "BETA",
"description": "The launch stage of the Cloud Run Job V2. See <https://cloud.google.com/run/docs/about-features-categories> for additional details."
},
"env_from_secrets": {
"type": "object",
"title": "Environment Variables from Secrets",
"example": {
"ENV_VAR_NAME": {
"secret": "SECRET_NAME",
"version": "latest"
}
},
"description": "Environment variables to set from GCP secrets when starting a flow run."
},
"vpc_connector_name": {
"type": "string",
"title": "VPC Connector Name",
"description": "The name of the VPC connector to use for the Cloud Run job."
},
"service_account_name": {
"type": "string",
"title": "Service Account Name",
"example": "<mailto:service-account@example.iam.gserviceaccount.com|service-account@example.iam.gserviceaccount.com>",
"description": "The name of the service account to use for the task execution of Cloud Run Job. By default Cloud Run jobs run as the default Compute Engine Service Account. "
}
},
"definitions": {
"GcpCredentials": {
"type": "object",
"title": "GcpCredentials",
"properties": {
"project": {
"type": "string",
"title": "Project",
"description": "The GCP project to use for the client."
},
"service_account_file": {
"type": "string",
"title": "Service Account File",
"format": "path",
"description": "Path to the service account JSON keyfile."
},
"service_account_info": {
"type": "object",
"title": "Service Account Info",
"description": "The contents of the keyfile as a dict."
}
},
"description": "Block used to manage authentication with GCP. Google authentication is handled via the `google.oauth2.\nSee Google's [Authentication documentation](<https://cloud.google.com/docs/authentication#service-accounts>)\nfor details on inference and recommended authentication patterns.",
"secret_fields": [
"service_account_info.*"
],
"block_type_slug": "gcp-credentials",
"block_schema_references": {}
},
"OidcGcpCredentials": {
"type": "object",
"title": "OidcGcpCredentials",
"required": [
"project",
"workload_identity_audience",
"service_account_email"
],
"properties": {
"project": {
"type": "string",
"title": "Project",
"description": "The GCP project to use for the client."
},
"service_account_email": {
"type": "string",
"title": "Service Account Email",
"description": "Service account email to impersonate (e.g., '<mailto:prefect-runner@my-project.iam.gserviceaccount.com|prefect-runner@my-project.iam.gserviceaccount.com>')"
},
"workload_identity_audience": {
"type": "string",
"title": "Workload Identity Audience",
"description": "The audience URL from your workload identity provider."
}
},
"description": "Block used to manage authentication with GCP using OIDC Workload Identity Federation."
}
},
"description": "Default variables for the Cloud Run V2 Push Pool.\n\nThe schema for this class is used to populate the `variables` section of the\ndefault base job template."
},
"job_configuration": {
"env": "{{ env }}",
"name": "{{ name }}",
"labels": "{{ labels }}",
"region": "{{ region }}",
"command": "{{ command }}",
"timeout": "{{ timeout }}",
"job_body": {
"client": "prefect",
"template": {
"template": {
"timeout": "{{ timeout }}",
"vpcAccess": {
"connector": "{{ vpc_connector_name }}"
},
"containers": [
{
"env": [],
"args": "{{ args }}",
"image": "{{ image }}",
"command": "{{ command }}",
"resources": {
"limits": {
"cpu": "{{ cpu }}",
"memory": "{{ memory }}"
}
}
}
],
"maxRetries": "{{ max_retries }}",
"serviceAccount": "{{ service_account_name }}"
}
},
"launchStage": "{{ launch_stage }}"
},
"keep_job": "{{ keep_job }}",
"credentials": "{{ credentials }}",
"env_from_secrets": "{{ env_from_secrets }}"
}
}Marvin
10/06/2025, 7:36 PMf dbt-run-24h-{datetime now() strftime( %Y-%m-%d-%H-%M-%S )} so I removed the problematic characters I'm still getting the same warnings on flow runs I'm using prefect==3 4 12 prefect-gcp==0 6 10 Here is the base job template ```{ variables { type object required [ credentials ] properties { cpu { type string title CPU default 1000m pattern ^(\\d*000)m$ description The CPU to allocate to the Cloud Run job } env { type object title Environment Variables default { prefect_var_env test prefect_var_region europe-north1 prefect_var_gcp_account_id project-name } description Environment variables to set when starting a flow run additionalProperties { type string } } args { type array items { type string } title Args description The arguments to pass to the Cloud Run Job V2's entrypoint command } name { type string title Name description Name given to created infrastructure } image { type string title Image Name default prefecthq/prefect 3 4 12-python3 12 description The image to use for the Cloud Run job If not provided the latest Prefect image will be used } labels { type object title Labels description Labels applied to created infrastructure additionalProperties { type string } } memory { type string title Memory default 512Mi example 512Mi pattern ^\\d+(? G|Gi|M|Mi)$ description The memory to allocate to the Cloud Run job along with the units which could be G Gi M Mi } region { type string title Region default europe-north1 example europe-north1 description The region in which to run the Cloud Run job } command { type string title Command description The command to use when starting a flow run In most cases this should be left blank and the command will be automatically generated } timeout { type integer title Job Timeout default 3600 maximum 86400 description The length of time that Prefect will wait for a Cloud Run Job to complete before raising an exception (maximum of 86400 seconds 1 day) exclusiveMinimum 0 } keep_job { type boolean title Keep Job After Completion default false description Keep the completed Cloud run job on Google Cloud Platform } credentials { anyOf [ { $ref #/definitions/GcpCredentials } { $ref #/definitions/OidcGcpCredentials } ] title GCP Credentials default { $ref { block_document_id redacted-uuid-reference-id } } description The GCP Credentials used to initiate the Cloud Run Job service_account_info is required } max_retries { type integer title Max Retries default 0 description The number of times to retry the Cloud Run job } launch_stage { enum [ ALPHA BETA GA DEPRECATED EARLY_ACCESS PRELAUNCH UNIMPLEMENTED LAUNCH_TAG_UNSPECIFIED ] type string title Launch Stage default BETA description The launch stage of the Cloud Run Job V2 See <https //cloud google com/run/docs/about-features-categories> for additional details } env_from_secrets { type object title Environment Variables from Secrets example { ENV_VAR_NAME { secret SECRET_NAME version latest } } description Environment variables to set from GCP secrets when starting a flow run } vpc_connector_name { type string title VPC Connector Name description The name of the VPC connector to use for the CloudJussi Kämäräinen
10/06/2025, 7:37 PMf"dbt-run-24h-{datetime.now().strftime("%Y-%m-%d-%H-%M-%S")}" so I removed the problematic characters. I'm still getting the same warnings on flow runs.
I'm using:
prefect==3.4.12
prefect-gcp==0.6.10Marvin
10/06/2025, 7:37 PMMarvin
10/06/2025, 7:45 PMflow_run_name=...), in code at runtime, or as part of a deployment name pattern?
Important context for Prefect 3.4.12 + prefect-gcp 0.6.10:
- There is no `name_template`/`run_name_template` argument on .deploy(). In Prefect 3, dynamic flow run names are set on the flow itself via the @flow decorator’s flow_run_name parameter.
- GCP workers (Cloud Run/GKE) generate a job/resource name and will “slugify” it (lowercase, replace invalid characters, truncate) using prefect_gcp.utilities.slugify_name. If the worker has to change the name, it logs a warning.
- Those warnings can be triggered by ANY of the names used to compose the job name, not just your flow run name. In practice, underscores, spaces, uppercase, and long names in your deployment name or flow name commonly cause the warnings even when the run name looks clean.
What to try/check
1) Ensure you’re setting the run name on the flow (and not expecting .deploy() to do it):
from datetime import datetime
from prefect import flow
@flow(flow_run_name=lambda: f"dbt-run-24h-{datetime.utcnow().strftime('%Y-%m-%d-%H-%M-%S')}")
def my_flow():
...
- Using a callable avoids any templating braces ending up in the final string and guarantees the final run name only includes the characters you return.
2) Check your other names that may be used in the worker’s job name:
- Deployment name
- Flow name (function name or @flow(name=...))
- Work pool and work queue names
Make sure they only use lowercase letters, numbers, and dashes and are reasonably short. Avoid underscores and spaces. Example:
# Good
name="dbt-daily-run"
# Risky (triggers slugify + warning)
name="DBT Daily_Run"
3) Keep names short
- The GCP worker will truncate names with slugify_name (default max length is limited) and can warn when truncating/sanitizing. Keep names reasonably short to avoid truncation warnings.
4) Turn on debug to see what’s being sanitized
- Run your worker with a higher log level to see the exact “before” and “after” name:
export PREFECT_LOGGING_LEVEL=DEBUG
# then start your worker as usual
This will usually reveal whether it’s your deployment name, flow name, or run name being altered.
Reference docs
- GCP worker guide: prefect-gcp worker guide
- Custom metadata and run names: Custom metadata and naming
If you can share:
- the exact warning text,
- your worker type,
- the flow decorator snippet (where you set flow_run_name),
- and your deployment name/work pool name,
I can identify which name is triggering the warning and suggest a precise fix.Jussi Kämäräinen
10/07/2025, 8:09 AMcloud-run:push workpool to cloud-run-v2:push workpool. The flow run executions start as intended but they give warning:
Unable to check status of flow run: <HttpError 404 when requesting <https://run.googleapis.com/v2/projects/project-name/locations/europe-north1/jobs/prefect-dbt-run-24h-2025-10-07-06-49-54/executions/prefect-sceptical-ocelot-5mwsr?alt=json> returned "Resource 'prefect-sceptical-ocelot-5mwsr' of kind 'EXECUTION' in region 'europe-north1' in project 'project-name' with parent 'prefect-dbt-run-24h-2025-10-07-06-49-54' does not exist.".
I have confirmed the related job prefect-sceptical-ocelot-5mwsr exists in Cloud Run job pool and executes without warnings in Cloud Run logs. So the warning only appears in Prefect UI when checking flow run logs.
I'm using:
prefect==3.4.12
prefect-gcp==0.6.10Marvin
10/07/2025, 8:09 AMMarvin
10/07/2025, 8:11 AMgcloud run jobs list --region europe-north1 --project project-name
- Check executions under each suspected job
gcloud run jobs executions list --job prefect-dbt-run-24h-2025-10-07-06-49-54 --region europe-north1 --project project-name
gcloud run jobs executions list --job prefect-sceptical-ocelot-5mwsr --region europe-north1 --project project-name
You should see executions under the sceptical-ocelot job, and none under the dbt-run job name. That will confirm the mismatch.
Why this happens
- In Cloud Run v2, a Job resource has Executions created under it when run.
- With prefect-gcp 0.6.10 on push pools, there appears to be a mismatch in how the job name and execution identifier are recorded and later used to poll status. Your custom flow run name (prefect-dbt-run-24h-...) is being used where the job name should be, while the friendly slug (prefect-sceptical-ocelot-...) is used as the execution id — reversed compared to what Cloud Run expects.
Recommended next steps
- Try upgrading to the latest prefect-gcp and Prefect 3.x. There have been steady improvements to workers and status polling:
- Prefect GCP worker guide: GCP worker guide
- Prefect how-to on naming/metadata: Custom metadata and naming
- As a temporary workaround, if you need to eliminate the warning immediately:
- Consider switching back to the cloud-run:push pool until you can upgrade.
- Or, set a consistent job name via your work pool’s default variables to ensure the actual Job name matches what Prefect is polling. If you have a “name” or “name_prefix” field for Cloud Run v2 jobs in your work pool default variables, try aligning it with your flow run name template (lowercase, dashes only, short). This may avoid the swapped-name condition. If you share a screenshot of your Cloud Run v2 pool’s default variables, I can suggest exact values.
To help us get a precise fix
If you can share:
- The output of the gcloud commands above (just the names),
- Your work pool type exactly (“cloud-run-v2:push”) and pool default variables (redact secrets),
- Your flow decorator snippet where you set flow_run_name,
- The deployment name you’re using,
I can reproduce and open a targeted bug with the prefect-gcp team. The runs are healthy; this is a polling/lookup issue that we should correct so the warning goes away.Jussi Kämäräinen
10/07/2025, 8:24 AMtheta848-alphecca-quadrant that was renamed automatically in the UI to dbt-run-24h-2025-10-07-08-19-00.
I run the debuggin gcloud commands and got:
JOB REGION LAST RUN AT CREATED CREATED BY
prefect-theta848-alphecca-quadrant europe-north1 2025-10-07 08:18:02 UTC 2025-10-07 08:17:57 UTC <mailto:service-prefect-cr-wp-test@sok-gdw-main-test.iam.gserviceaccount.com|service-prefect-cr-wp-test@sok-gdw-main-test.iam.gserviceaccount.com>
And when checking the executions under that name, I get:
JOB EXECUTION REGION RUNNING COMPLETE CREATED RUN BY
… prefect-theta848-alphecca-quadrant prefect-theta848-alphecca-quadrant-7n94g europe-north1 1 0 / 1 2025-10-07 08:18:02 UTC <mailto:service-prefect-cr-wp-test@sok-gdw-main-test.iam.gserviceaccount.com|service-prefect-cr-wp-test@sok-gdw-main-test.iam.gserviceaccount.com>
The error is now:
Unable to check status of flow run: <HttpError 404 when requesting <https://run.googleapis.com/v2/projects/project-name/locations/europe-north1/jobs/prefect-dbt-run-24h-2025-10-07-08-19-00/executions/prefect-theta848-alphecca-quadrant-7n94g?alt=json> returned "Resource 'prefect-theta848-alphecca-quadrant-7n94g' of kind 'EXECUTION' in region 'europe-north1' in project 'project-name' with parent 'prefect-dbt-run-24h-2025-10-07-08-19-00' does not exist.".
Does that mean that the flow and execution names are mixed? How to fix this?Marvin
10/07/2025, 8:24 AMMarvin
10/07/2025, 8:26 AMprefect-theta848-alphecca-quadrant
- Actual Execution: prefect-theta848-alphecca-quadrant-7n94g
- Prefect is polling: parent Job prefect-dbt-run-24h-2025-10-07-08-19-00 with execution prefect-theta848-alphecca-quadrant-7n94g
So Prefect is using the current flow run name as the Cloud Run Job parent, but the worker created the Job under the original/default slug name. That explains the 404 even though the run is healthy.
Why this is happening
- In Cloud Run v2, executions are nested under a specific Job.
- With CloudRun v2 push, there’s a mismatch between the name Prefect stores/uses for polling and the name actually used to create the Cloud Run Job when your flow run is renamed at start (e.g., via flow_run_name).
Workarounds to stop the warnings
- Easiest: temporarily remove your custom flow_run_name so the flow run keeps the default slug (e.g., theta848-alphecca-quadrant). That should make the polled Job parent match the actual Job and eliminate the 404 warnings.
- Alternatively, roll back to the cloud-run:push work pool until a fix is available for v2.
- If you rely on custom names, try updating to the latest Prefect 3.x and prefect-gcp versions; there have been ongoing worker/polling improvements. GCP worker guide: GCP worker guide. Custom metadata and naming: Custom metadata and naming.
Next steps I can take
- If you can confirm that removing flow_run_name makes the warning disappear, I’ll summarize this as a reproducible bug and open a thread with the team. If you’re okay with it, I can also create a GitHub Discussion to track it publicly so others can follow the fix.
- If you prefer to keep the custom run name, I can help you test a few worker variable tweaks — but given Cloud Run v2’s naming handling, the reliable immediate fix is to avoid renaming the flow run until the polling logic stores/uses the actual Job name created by the worker.
Want me to file a summary for the Prefect team with your reproduction steps and the gcloud outputs you shared?