Here’s a concise guide to Client Metrics Settings in Prefect 3.x. What it is - Controls whether the Prefect client (including workers/flow runs) exposes a local metrics endpoint, and on which port. - Disabled by default. Fields and defaults - enabled (bool, default: false) — turn the metrics endpoint on/off - port (int, default: 4201) — TCP port to bind for the local metrics endpoint Docs: - Settings reference: ClientMetricsSettings (settings ref) - Class API: ClientMetricsSettings (class) How to configure - Via environment variables (highest precedence): -

PREFECT_CLIENT_METRICS_ENABLED=true

(alias:

PREFECT_CLIENT_ENABLE_METRICS=true

) -

PREFECT_CLIENT_METRICS_PORT=8888

- Via TOML (e.g., prefect.toml or pyproject.toml):

[client.metrics]
  enabled = true
  port = 4201

- Programmatically (inspect/temporarily override):

from prefect.settings import get_current_settings, temporary_settings

  s = get_current_settings()
  print(s.client.metrics.enabled, s.client.metrics.port)

  with temporary_settings({"client.metrics.enabled": True, "client.metrics.port": 8888}):
      # code that runs with metrics enabled on port 8888
      ...

Behavior/notes - When enabled, Prefect starts a local metrics endpoint on the configured port. - Use cases: integrate with your observability stack by scraping this endpoint (e.g., with a Prometheus/OTel collector). - Port conflicts: if multiple processes run on the same host, assign unique ports or disable metrics for some. - Containers/managed platforms: - The endpoint binds to the runtime’s network namespace. To scrape it from outside the container, expose/forward the port or run an in-pod/sidecar collector that can access localhost. - For Vertex AI work pools, set the env vars either in the work pool defaults or on deployments so your Vertex jobs start with metrics enabled; scraping typically requires a sidecar/collector within the same job or network access to the pod. More on settings mechanics and precedence: Settings & profiles Would you like a quick example of enabling this for your VertexAI work pool (where to set env vars and options for scraping)?

Short answer: yes—the metrics endpoint is started inside each flow-run process when

client.metrics.enabled

is true. It binds to the runtime’s localhost on the configured port (default 4201). That means it’s reachable from within the container/VM that’s running the flow, not from the outside. In dynamic/serverless runtimes like Cloud Run Jobs, you generally cannot scrape that endpoint externally because: - Jobs don’t expose inbound ports for scraping - The endpoint binds to localhost - Targets are ephemeral and IPs are dynamic Common patterns that work 1) In-container or sidecar collector (recommended) - Run an OpenTelemetry Collector or Prometheus inside the same job to scrape

127.0.0.1:<port>

and export metrics to your backend (Cloud Monitoring, Prometheus Remote Write, Grafana Cloud, etc.). - If the platform supports multiple containers per job, use a sidecar collector. If not, bundle the collector binary into your image and start it alongside your flow process. Example otel-collector config (scrape localhost and export):

receivers:
  prometheus:
    config:
      scrape_configs:
      - job_name: "prefect-client"
        scrape_interval: 15s
        static_configs:
        - targets: ["127.0.0.1:4201"]

exporters:
  googlecloud: {}             # Cloud Monitoring
  # or Prometheus remote write:
  # prometheusremotewrite:
  #   endpoint: <https://your-prom-remote-write-endpoint/api/v1/write>
  # or generic OTLP:
  # otlphttp:
  #   endpoint: <https://otlp.your-backend:4318>

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      exporters: [googlecloud]  # or prometheusremotewrite / otlphttp

Example entrypoint for a single-container Cloud Run Job:

#!/usr/bin/env bash
set -euo pipefail

# Start a local collector in the background to scrape localhost:4201
otelcol --config /etc/otel-collector-config.yaml &

# Enable Prefect client metrics for the flow run
export PREFECT_CLIENT_METRICS_ENABLED=true
export PREFECT_CLIENT_METRICS_PORT=4201

# Start your flow
python -m your_flow_entrypoint

2) End-of-job snapshot/push (least preferred) - At the end of the run, curl the metrics endpoint and push the snapshot to a Pushgateway or your metrics API. This misses time-series over the run’s lifetime but can be sufficient for some use cases.

#!/usr/bin/env bash
set -euo pipefail

export PREFECT_CLIENT_METRICS_ENABLED=true
export PREFECT_CLIENT_METRICS_PORT=4201

python -m your_flow_entrypoint

# Snapshot and push (example: Pushgateway)
curl -s <http://127.0.0.1:4201/metrics> | curl --data-binary @- <http://pushgateway:9091/metrics/job/prefect_client>

Notes for Vertex AI / Cloud Run - The endpoint is only inside the runtime; don’t expect external scraping to work. - For Vertex AI work pools or Cloud Run Jobs, put the env vars in your work pool defaults or deployment so each job enables the endpoint. - If you can’t run a sidecar, embed a lightweight collector in your image (as shown above). References - Client metrics settings: ClientMetricsSettings (settings ref) - Class API: ClientMetricsSettings (class) - Telemetry bootstrap (for broader context on OTel in Prefect): prefect.telemetry.bootstrap If you can share where you want the metrics to land (Prometheus, Cloud Monitoring, Datadog, etc.), I can tailor a working collector config for Cloud Run Jobs or Vertex AI.

Short answer: there isn’t a published, fixed list of client-exposed metrics in the Prefect 3.x docs right now. The set is derived from our OpenTelemetry-based instrumentation and may evolve. It typically includes runtime/process metrics and instrumented client activity, but exact names/labels can vary by version and environment. Practical ways to see what you get - Locally: 1) Enable metrics

export PREFECT_CLIENT_METRICS_ENABLED=true
     export PREFECT_CLIENT_METRICS_PORT=4201

2) Run a simple flow that stays alive for a bit, then in another shell:

curl -s <http://127.0.0.1:4201/metrics> | head -n 100

- Cloud Run Jobs / other ephemeral: - Add a final step to dump the endpoint to logs so you can review the exact metric names/labels from a real run:

curl -s <http://127.0.0.1:4201/metrics> | sed -n '1,200p'

- Or, run an in-job/sidecar collector (Prometheus or OTel Collector), scrape 127.0.0.1:<port>, and inspect what it ingests. Notes - Client metrics are separate from Prefect’s server-side workflow metrics (e.g., successes, duration, lateness) which are derived from events. - The endpoint is generally Prometheus-format at http://localhost:&lt;port&gt;/metrics when

client.metrics.enabled

is true. - Because the set is not guaranteed stable across releases, treating it as discoverable at runtime is best practice. If you can share a snippet of your

/metrics

output from a run, I’m happy to help enumerate the metric names and suggest dashboards/alerts for Cloud Run or Vertex AI. References - Settings: ClientMetricsSettings (settings ref) - Class API: ClientMetricsSettings (class)