Hey all, has anyone tried setting up a Grafana dashboard or Prometheus datasource for Prefect 2? I saw that there's this closed issue about it, and this somewhat related open issue about it, but nothing actually implemented. I also found this community-made Prometheus exporter, but it hasn't been touched since June 2022 and looks like its for Prefect v1. Does anyone know of any other existing Prometheus exporters or anything that would be helpful for collecting Prefect metrics? I know I can also make my own using the Prefect API, but I was just hoping there was an already-existing solution somewhere.

HI Nick - I wrote the one here you can use as a template - https://github.com/PrefectHQ/prefect-recipes/tree/main/prefect-v1-legacy/devops/monitoring

Gotcha, thank you!

Hey @Nick Hoffmann, what did you end up doing for exporting metrics s.t. it can be nicely displayed on Grafana? Did you end up using the custom Prometheus exporter?

I did not end up making one, that was more work than was worth for the time being. Instead I just push metrics to the prometheus pushgateway from each flow run to get the basics, like flow start time, duration, status, etc. I toyed with the idea of getting more in depth with it and interacting with the REST API from the flow itself to get closer to mimicking an actual exporter, but I turned that down for the sake of simplicity

hm, I see that makes sense. Thanks!

I'm looking forward to implementing prometheus to export metrics so I can monitor Prefect. @Christopher Boyd thank you for writing the exporter... do you have any idea of what changes we need to do to make the exporter compatible with Prefect 2? I think I can spend some time to have the exporter updated. Would this repo be enough to have basic monitoring?

That’s a good start - for the most part, it would need to be rewritten from graphql to rest