Great question — and you’re right to consider the Cloud retention constraints up-front. Short answer: - Prefect’s Kubernetes jobs/pods are labeled with Prefect metadata (flow run ID, deployment ID/name, etc.). If the K8s Job/Pod still exists, you can extract those labels and directly identify the deployment — even if the flow run is older than Cloud’s 7‑day retention. - If the K8s objects are gone, your best bet is your worker logs or cluster/audit logs where those labels and the flow-run → job name mapping are recorded. Here’s how to backtrack from a pod name/UID to the deployment. 1) If the pod still exists - Get all Prefect labels from the pod:

kubectl get pod <pod-name> -n <namespace> -o=jsonpath='{.metadata.labels}'

- Useful labels you should see on pods created by the Kubernetes worker (3.x): - prefect.io/flow-run-id - prefect.io/flow-run-name - prefect.io/flow-id - prefect.io/flow-name - prefect.io/deployment-id - prefect.io/deployment-name - prefect.io/work-pool-id - prefect.io/work-pool-name - prefect.io/worker-name - prefect.io/version - prefect.io/deployment-updated Example evidence (with full label list) from a recent report: prefecthq/prefect#18835 - Extract specific values:

kubectl get pod <pod-name> -n <ns> -o=jsonpath='{.metadata.labels["<http://prefect.io/deployment-id|prefect.io/deployment-id>"]}{"\n"}{.metadata.labels["<http://prefect.io/deployment-name|prefect.io/deployment-name>"]}{"\n"}{.metadata.labels["<http://prefect.io/flow-name|prefect.io/flow-name>"]}{"\n"}{.metadata.labels["<http://prefect.io/flow-run-id|prefect.io/flow-run-id>"]}{"\n"}'

- Once you have the flow name and deployment name, you can inspect the deployment:

prefect deployment inspect "<flow-name>/<deployment-name>"

- If you only have the deployment ID (and not the names), use the Python client to fetch it:

import anyio
from prefect import get_client

DEPLOYMENT_ID = "00000000-0000-0000-0000-000000000000"

async def main():
    async with get_client() as client:
        dep = await client.read_deployment(DEPLOYMENT_ID)
        print(dep.id, dep.name, dep.flow_name)

anyio.run(main)

Tip: If you only know the pod UID and not the name:

kubectl get pods -n <ns> --field-selector metadata.uid=<pod-uid>

2) If the pod is gone but the Job still exists - Get the owning Job name from the Pod ownerReferences (when the pod exists), or if you already know it, dump the Job’s labels:

kubectl get job <job-name> -n <ns> -o=jsonpath='{.metadata.labels}'

The same Prefect labels are typically present on the Job object as well. 3) If both Job and Pod are gone (TTL cleanup) - You won’t find them in the cluster. Use one of: - Prefect worker logs: Kubernetes workers log both the Prefect flow run ID and the created Job name when they submit, e.g. “Submitting flow run <id> … Creating Kubernetes job <job-name>”. - Cluster/audit logs: If you ship Kubernetes API audit logs or Pod metadata to a log sink (CloudWatch/Stackdriver/ELK), search for the pod/job name and look for the labels above. - Your centralized application logs: Flow run logs often include the run ID and flow/deployment names early in execution. 4) Opening the deployment in Prefect Cloud - With a deployment ID from labels/logs, you can open the deployment directly in the UI: - URL pattern: https://app.prefect.cloud/&lt;account-id&gt;/&lt;workspace-id&gt;/deployments/&lt;deployment-id> - Or query it via the Python client as shown above. Notes and best practices (given 7‑day flow run retention) - Keep K8s resources long enough: Set or increase the Job TTL so Jobs/Pods are still around when you need to investigate. - In Prefect 3.x, set

job_variables.ttl_seconds_after_finished

on the deployment (via

prefect.yaml

or

.to_deployment(..., job_variables=...)

) to a value > 30 days, or null to retain indefinitely. - Docs: Customize job variables

- Rely on labels for traceability: The Prefect worker applies the

<http://prefect.io/*|prefect.io/*>

labels shown above to Jobs/Pods, which is the most reliable breadcrumb trail back to the deployment. This works regardless of Cloud run retention. - Centralize logs/metadata: Ship K8s audit and worker logs to a log store so you can recover flow-run/deployment IDs even if K8s TTL has cleaned up the objects. - Kubernetes guide for reference: Kubernetes deployment infrastructure Quick checklist for your 30‑day lookback case - If you still have the pod: 1) Run

kubectl get pod <pod> -o=jsonpath='{.metadata.labels}'

2) Read

<http://prefect.io/deployment-id|prefect.io/deployment-id>

or

<http://prefect.io/deployment-name|prefect.io/deployment-name>

+

<http://prefect.io/flow-name|prefect.io/flow-name>

3) Inspect the deployment:

prefect deployment inspect "<flow-name>/<deployment-name>"

or use the Python client with the deployment ID - If the pod is gone: 1) Search worker/cluster logs for the pod or job name, extract the

<http://prefect.io/*|prefect.io/*>

labels or the flow run ID 2) Use the deployment ID or names to inspect/open the deployment as above Do you still have the Job/Pod objects in the cluster, or will we need to go via logs? If you share the pod name/UID pattern and your namespace, I can give you the exact

kubectl

commands to run.

Thanks for the follow-up — what you’re seeing are Google “resource labels” from Cloud Logging, which are not the same as the Kubernetes metadata labels that Prefect adds to Jobs/Pods. Prefect’s labels will not show up under `resource.labels`; they appear as Kubernetes pod labels and in Cloud Logging are exposed either: - as promoted labels like

labels."k8s-pod/<domain>/<key>"

, or - inside the structured payload under something like

jsonPayload.k8s.pod.labels."<domain>/<key>"

What Prefect auto-applies on K8s Jobs/Pods - The Kubernetes worker applies a set of

<http://prefect.io/*|prefect.io/*>

labels to the Job and Pod it creates, including: -

<http://prefect.io/flow-run-id|prefect.io/flow-run-id>

-

<http://prefect.io/flow-run-name|prefect.io/flow-run-name>

-

<http://prefect.io/flow-id|prefect.io/flow-id>

-

<http://prefect.io/flow-name|prefect.io/flow-name>

-

<http://prefect.io/deployment-id|prefect.io/deployment-id>

-

<http://prefect.io/deployment-name|prefect.io/deployment-name>

-

<http://prefect.io/work-pool-id|prefect.io/work-pool-id>

-

<http://prefect.io/work-pool-name|prefect.io/work-pool-name>

-

<http://prefect.io/worker-name|prefect.io/worker-name>

-

<http://prefect.io/version|prefect.io/version>

-

<http://prefect.io/deployment-updated|prefect.io/deployment-updated>

- Example (from a recent report showing these exact labels on a job pod): prefecthq/prefect#18835 - Reference docs: - Kubernetes deployment infrastructure - Customize job variables How to verify in-cluster (recommended sanity check) - Inspect pod labels directly:

kubectl get pod <pod-name> -n <namespace> -o=jsonpath='{.metadata.labels}'

- Or extract specific Prefect labels:

kubectl get pod <pod-name> -n <ns> -o=jsonpath='{.metadata.labels["<http://prefect.io/deployment-id|prefect.io/deployment-id>"]}{"\n"}{.metadata.labels["<http://prefect.io/deployment-name|prefect.io/deployment-name>"]}{"\n"}{.metadata.labels["<http://prefect.io/flow-run-id|prefect.io/flow-run-id>"]}{"\n"}'

- You can also list all jobs/pods that have a flow-run label:

kubectl get jobs -A -l <http://prefect.io/flow-run-id|prefect.io/flow-run-id>
kubectl get pods -A -l <http://prefect.io/flow-run-id|prefect.io/flow-run-id>

How to find these in Google Cloud Logging - Google rewrites label keys for indexing. For example, the Kubernetes label

<http://batch.kubernetes.io/controller-uid|batch.kubernetes.io/controller-uid>

appears as

labels."k8s-pod/batch_kubernetes_io/controller-uid"

in Cloud Logging (note dots -> underscores in the domain). - By analogy,

<http://prefect.io/deployment-id|prefect.io/deployment-id>

typically appears as: - Promoted label:

labels."k8s-pod/prefect_io/deployment-id"

- Or in the structured payload:

jsonPayload.k8s.pod.labels."<http://prefect.io/deployment-id|prefect.io/deployment-id>"

- Try these queries in Logs Explorer (adjust the resource type to your environment):

labels."k8s-pod/prefect_io/deployment-id"=*

or

jsonPayload.k8s.pod.labels."<http://prefect.io/deployment-id|prefect.io/deployment-id>"=*

- If your logs don’t show Kubernetes labels under

labels.*

, open a single log entry and browse the structured fields to find

k8s.pod.labels

. In some setups, only a subset of pod labels are promoted to

labels.*

while the full set is still available in the structured payload. If you still don’t see Prefect labels - The pod may not have been created by a Prefect Kubernetes worker (e.g., a custom Job template that didn’t merge default labels). - The Job/Pod was heavily customized and metadata.labels were overridden. - Your GKE logging configuration may not be including Kubernetes labels at the level you expect; check if labels are present in the structured payload even if not promoted to

labels.*

. - Less likely but possible: older worker or

prefect-kubernetes

integration version differences. If you share your Prefect version and

prefect-kubernetes

version, I can confirm behavior. Fallbacks - Environment variables inside the pod often include `PREFECT__FLOW_RUN_ID`:

kubectl get pod <pod> -n <ns> -o=jsonpath='{.spec.containers[*].env}'

Search the output for

PREFECT__FLOW_RUN_ID

. - Worker logs: look for lines like “Submitting flow run <id>… Creating Kubernetes job <job-name>”, which tie job/pod names to flow run/deployment. If you can share: - One example pod name and namespace

- Your Prefect version (

prefect version

) and

prefect-kubernetes

version I’ll give you an exact `kubectl`/Logs Explorer query that should surface the deployment ID quickly.

Thanks for sharing the raw GKE log — that helps narrow this down. What your log shows - It’s a container log for

container_name="prefect-job"

in pod

tunneling-rhino-c8bbx-2kkr5

, owned by Job

tunneling-rhino-c8bbx

. - Only a small set of Kubernetes labels were promoted into

labels.*

by Cloud Logging: -

k8s-pod/job-name

,

k8s-pod/batch_kubernetes_io/controller-uid

, etc. - There are no Prefect-specific labels in this record. What that means - From this single log entry alone, you cannot recover the Prefect deployment ID — GKE didn’t include Prefect’s pod labels in the promoted

labels.*

set for this entry. - However, Prefect’s Kubernetes worker typically applies Prefect labels to Jobs/Pods (e.g.,

<http://prefect.io/deployment-id|prefect.io/deployment-id>

,

<http://prefect.io/deployment-name|prefect.io/deployment-name>

,

<http://prefect.io/flow-run-id|prefect.io/flow-run-id>

, etc.). Those can still be present in: - other log entries for the same pod (as promoted labels), or - the structured payload (e.g.,

jsonPayload.k8s.pod.labels."<http://prefect.io/deployment-id|prefect.io/deployment-id>"

), or - the live K8s resources (if they haven’t been cleaned up), or - the Prefect worker logs at submission time. How to answer your original question given this log Use these steps in order until you get a hit: 1) Search Cloud Logging for Prefect labels for this pod - Try promoted label variants (GKE rewrites “prefect.io” → “prefect_io” in

labels.k8s-pod/...

):

resource.type="k8s_container"
resource.labels.namespace_name="prefect"
resource.labels.pod_name="tunneling-rhino-c8bbx-2kkr5"
labels."k8s-pod/prefect_io/deployment-id"=*

- If nothing, inspect the structured payload instead:

resource.type="k8s_container"
resource.labels.namespace_name="prefect"
resource.labels.pod_name="tunneling-rhino-c8bbx-2kkr5"
jsonPayload.k8s.pod.labels."<http://prefect.io/deployment-id|prefect.io/deployment-id>"=*

- Also try:

labels."k8s-pod/prefect_io/flow-run-id"=* OR jsonPayload.k8s.pod.labels."<http://prefect.io/flow-run-id|prefect.io/flow-run-id>"=*

If found, you’ll have

deployment-id

and/or

deployment-name

directly. 2) If the pod or job still exists in the cluster - Pull labels straight from Kubernetes (these are the authoritative source):

kubectl get pod tunneling-rhino-c8bbx-2kkr5 -n prefect -o=jsonpath='{.metadata.labels}'
kubectl get job tunneling-rhino-c8bbx -n prefect -o=jsonpath='{.metadata.labels}'

Look for keys like: -

<http://prefect.io/deployment-id|prefect.io/deployment-id>

-

<http://prefect.io/deployment-name|prefect.io/deployment-name>

-

<http://prefect.io/flow-run-id|prefect.io/flow-run-id>

-

<http://prefect.io/flow-run-name|prefect.io/flow-run-name>

-

<http://prefect.io/flow-name|prefect.io/flow-name>

-

<http://prefect.io/work-pool-name|prefect.io/work-pool-name>

Evidence that these labels are applied on recent versions: prefecthq/prefect#18835 3) If the K8s resources are gone (TTL cleanup) - Search the Prefect worker logs around the pod’s timestamp. The worker logs when it submits a flow run and includes both the flow run ID and the created Job name, for example: - “Submitting flow run 'cb91b9ce-...' … Creating Kubernetes job 'tunneling-rhino-c8bbx'” - This gives you the flow run ID → which often leads to the deployment ID/name (if still retained). - If Prefect Cloud flow run retention is only 7 days and you’re looking 30 days back, you won’t be able to query the run in Cloud — so keep relying on the worker logs and/or K8s labels captured in your log sink. 4) Once you have either deployment ID or names - Inspect in CLI or open in UI:

prefect deployment inspect "<flow-name>/<deployment-name>"

Or open: https://app.prefect.cloud/&lt;account-id&gt;/&lt;workspace-id&gt;/deployments/&lt;deployment-id> Why you didn’t see Prefect labels in this log - GKE only promotes a subset of labels into the top-level

labels.*

for a given log line. Other labels may be present only in the structured payload. In many clusters, the Prefect labels show up as either: -

labels."k8s-pod/prefect_io/<key>"

, or -

jsonPayload.k8s.pod.labels."<http://prefect.io/<key>|prefect.io/<key>>"

- It’s also possible the pod didn’t have those labels (e.g., very old worker/integration, or a custom job template that replaced

metadata.labels

instead of merging).

Recommended hardening so you can always backtrack after 30 days - Make sure the worker’s Jobs/Pods carry the Prefect labels; test with a recent run:

kubectl get pods -A -l <http://prefect.io/flow-run-id|prefect.io/flow-run-id>

- Keep Jobs around long enough to investigate by setting TTL on jobs via job variables: -

ttl_seconds_after_finished

to >30 days or null - Docs: Customize job variables - Ensure your logging pipeline preserves

k8s.pod.labels

fields. If you’re using GKE “Autopilot logging”, verify that Kubernetes labels are included in the ingested payload. If you can share: - Prefect version (

prefect version

) and

prefect-kubernetes

version - Whether you’re using a custom baseJobTemplate - One pod that still exists in-cluster …I can confirm whether Prefect labels are being applied and give you the exact Logs Explorer query for your setup. Reference: Kubernetes worker behavior and job/pod metadata - Kubernetes deployment infrastructure - Example showing Prefect labels applied on pods/jobs: prefecthq/prefect#18835

Got it. Here’s how you can map a flow run ID (older than 14 days) to its deployment ID given Cloud retention limits. What’s possible depends on what you still have available: A) If Kubernetes resources still exist (Jobs/Pods not GC’d) - Pods/Jobs created by the Kubernetes worker are labeled with both the flow run ID and the deployment ID. - Search by the flow run ID label and read the deployment ID label:

kubectl get pods -A -l <http://prefect.io/flow-run-id=<FLOW_RUN_ID|prefect.io/flow-run-id=<FLOW_RUN_ID>> \
  -o=jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{" "}{.metadata.labels["<http://prefect.io/deployment-id|prefect.io/deployment-id>"]}{" "}{.metadata.labels["<http://prefect.io/deployment-name|prefect.io/deployment-name>"]}{"\n"}{end}'

kubectl get jobs -A -l <http://prefect.io/flow-run-id=<FLOW_RUN_ID|prefect.io/flow-run-id=<FLOW_RUN_ID>> \
  -o=jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{" "}{.metadata.labels["<http://prefect.io/deployment-id|prefect.io/deployment-id>"]}{" "}{.metadata.labels["<http://prefect.io/deployment-name|prefect.io/deployment-name>"]}{"\n"}{end}'

- Evidence of these labels on Kubernetes Jobs/Pods: - prefecthq/prefect#18835 - Kubernetes guide (3.x): Kubernetes deployment infrastructure B) If K8s objects are gone, but you have a log sink (e.g., GKE Cloud Logging) - GKE may not promote all pod labels into

labels.*

, but they’re often present either as promoted labels or in the structured payload. Try both: - Promoted label form (note the domain rewrite

prefect_io

):

resource.type="k8s_container"
labels."k8s-pod/prefect_io/flow-run-id"="<FLOW_RUN_ID>"

- Structured payload form:

resource.type="k8s_container"
jsonPayload.k8s.pod.labels."<http://prefect.io/flow-run-id|prefect.io/flow-run-id>"="<FLOW_RUN_ID>"

- Once you find a log entry with that flow run ID, read the accompanying deployment labels: -

labels."k8s-pod/prefect_io/deployment-id"

or

jsonPayload.k8s.pod.labels."<http://prefect.io/deployment-id|prefect.io/deployment-id>"

- You may also see

<http://prefect.io/deployment-name|prefect.io/deployment-name>

,

<http://prefect.io/flow-name|prefect.io/flow-name>

, etc. - If you primarily have worker logs, search for the flow run ID; workers log the Job name and often include metadata that lets you correlate back to the pod (which then carries the deployment labels). C) If neither K8s objects nor logs are available - With Cloud’s 7‑day flow run retention, Prefect Cloud won’t be able to return the flow run or its deployment for a 14+ day old run. - In that case, there isn’t a way to recover the deployment ID from Prefect after the fact. You’ll need an external source (K8s labels captured in logs, worker logs, or your own audit store). Hardening for the future (so you can always answer this) - Rely on the Prefect labels on Jobs/Pods: - Confirm they’re present on your cluster with:

kubectl get pods -A -l <http://prefect.io/flow-run-id|prefect.io/flow-run-id>

- Keep Jobs/Pods around long enough to investigate: - Set

ttl_seconds_after_finished

in

job_variables

to >30 days (or null). - Docs: Customize job variables - Persist the mapping at flow start: - Write a small on-start step that records

prefect.runtime.flow_run.id

,

prefect.runtime.deployment.id

, and

prefect.runtime.deployment.name

to a durable store (BigQuery/GCS/S3/DB).

from prefect import flow, get_run_logger
from prefect.runtime import flow_run, deployment

@flow
def my_flow():
    logger = get_run_logger()
    <http://logger.info|logger.info>(f"flow_run_id={flow_run.id} deployment_id={deployment.id} deployment_name={deployment.name}")
    # also write to your audit store here

If you can share the specific FLOW_RUN_ID and whether you still have K8s logs or resources, I can give you the exact kubectl or Logs Explorer query to pull the

<http://prefect.io/deployment-id|prefect.io/deployment-id>

for that run.