hi <@ULVA73B9P>, how can I install prefect-promet...
# ask-community
k
hi @Marvin, how can I install prefect-prometheus exporter on ec2 machine
m
thought for 399.0 seconds
Great question! Quick heads-up: there isn’t an official “prefect-prometheus-exporter” package you install directly on an EC2 instance. In Prefect 3.x, the recommended path for Prometheus metrics is to use OpenTelemetry (OTel). You run an OpenTelemetry Collector on your EC2 instance, have Prefect send OTLP metrics to it, and let Prometheus scrape the collector’s /metrics endpoint. Also relevant: there’s an open request to expose a native /metrics endpoint in Prefect, but it’s not available today — see issue #10673. If that aligns with what you need, here’s a simple way to set it up on EC2. Option A: Quick start with Docker (fastest) 1) Create an OTel Collector config on your EC2 instance (otel-collector.yaml):
Copy code
receivers:
  otlp:
    protocols:
      http:
      grpc:

exporters:
  prometheus:
    endpoint: 0.0.0.0:9464
    namespace: prefect

processors:
  batch:

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus]
2) Run the collector
Copy code
docker run -d --name otelcol \
  -p 4318:4318 -p 4317:4317 -p 9464:9464 \
  -v $(pwd)/otel-collector.yaml:/etc/otelcol/config.yaml \
  otel/opentelemetry-collector:latest
3) Point Prefect workers/flows at the collector (set env vars where you start your worker/flows)
Copy code
export OTEL_EXPORTER_OTLP_ENDPOINT=<http://localhost:4318>
export OTEL_METRICS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_RESOURCE_ATTRIBUTES="service.name=prefect-worker,service.instance.id=$(hostname)"
4) Configure Prometheus to scrape the collector Add a job in your prometheus.yml:
Copy code
scrape_configs:
  - job_name: 'prefect'
    static_configs:
      - targets: ['<EC2-IP>:9464']
Option B: Install the OTel Collector as a service (no Docker) - Follow the official OTel Collector install docs for your OS and use the same config above, then start/enable the service: - https://opentelemetry.io/docs/collector/ What metrics you’ll see - Prefect 3 uses OpenTelemetry under the hood; exporting metrics requires an OTel pipeline. You’ll get OTel-based metrics and anything you instrument in your flows. If you want specific orchestration metrics (e.g., number of running/failed flow runs) you can: - Emit custom metrics from your flows using the OpenTelemetry SDK or the Prometheus client library, or - Pull those stats from the Prefect API and expose them via a small custom exporter process. A couple of clarifying questions to fine-tune the setup: - Are you on Prefect Cloud or self-hosted Prefect server? - Which metrics do you specifically want in Prometheus (runtime app metrics, orchestration stats like flow/task counts, worker health, etc.)? - What OS is your EC2 machine running (Amazon Linux, Ubuntu, etc.) and do you prefer Docker or a native service install? If you were referring to a Helm chart named “prefect-prometheus-exporter,” that’s for Kubernetes (EKS) deployments, not plain EC2. On a non-Kubernetes EC2, the OTel Collector approach above is the recommended path.