Kiran
08/26/2025, 12:21 PMMarvin
08/26/2025, 12:22 PMMarvin
08/26/2025, 12:28 PMreceivers:
otlp:
protocols:
http:
grpc:
exporters:
prometheus:
endpoint: 0.0.0.0:9464
namespace: prefect
processors:
batch:
service:
pipelines:
metrics:
receivers: [otlp]
processors: [batch]
exporters: [prometheus]
2) Run the collector
docker run -d --name otelcol \
-p 4318:4318 -p 4317:4317 -p 9464:9464 \
-v $(pwd)/otel-collector.yaml:/etc/otelcol/config.yaml \
otel/opentelemetry-collector:latest
3) Point Prefect workers/flows at the collector (set env vars where you start your worker/flows)
export OTEL_EXPORTER_OTLP_ENDPOINT=<http://localhost:4318>
export OTEL_METRICS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_RESOURCE_ATTRIBUTES="service.name=prefect-worker,service.instance.id=$(hostname)"
4) Configure Prometheus to scrape the collector
Add a job in your prometheus.yml:
scrape_configs:
- job_name: 'prefect'
static_configs:
- targets: ['<EC2-IP>:9464']
Option B: Install the OTel Collector as a service (no Docker)
- Follow the official OTel Collector install docs for your OS and use the same config above, then start/enable the service:
- https://opentelemetry.io/docs/collector/
What metrics you’ll see
- Prefect 3 uses OpenTelemetry under the hood; exporting metrics requires an OTel pipeline. You’ll get OTel-based metrics and anything you instrument in your flows. If you want specific orchestration metrics (e.g., number of running/failed flow runs) you can:
- Emit custom metrics from your flows using the OpenTelemetry SDK or the Prometheus client library, or
- Pull those stats from the Prefect API and expose them via a small custom exporter process.
A couple of clarifying questions to fine-tune the setup:
- Are you on Prefect Cloud or self-hosted Prefect server?
- Which metrics do you specifically want in Prometheus (runtime app metrics, orchestration stats like flow/task counts, worker health, etc.)?
- What OS is your EC2 machine running (Amazon Linux, Ubuntu, etc.) and do you prefer Docker or a native service install?
If you were referring to a Helm chart named “prefect-prometheus-exporter,” that’s for Kubernetes (EKS) deployments, not plain EC2. On a non-Kubernetes EC2, the OTel Collector approach above is the recommended path.