Hey guys, we are currently evaluating prefect as a possibility to manage some of our workflows and it looks very promising. 🙂 We use prometheus to monitor all our services, so I want to monitor our flows as well. These run asynchroniously scheduled on local agents with dask executors and stored to s3 (as well as the flow results). Prometheus has a python_client https://github.com/prometheus/client_python, but I can't manage to integrate these metrics to our flows. I tried to include them in a task (say prometheus_monitor_task) as well as on callback, when a flow changes its state, but then I get the error, that the flow can't be pickled. How would you monitor your flows in addition to custom loggers? Is there any support/will there be support in the near future?

Hi @Julian! I'm not super familiar with Prometheus (reading up a bit now), but does this seem like something logging handlers might be able to help you with? https://docs.prefect.io/core/advanced_tutorials/custom-logs.html#adding-handlers

Thank you for your response. A post request to an external webserver might do the job. To follow up my question. How would you retrieve stats from flow runs/tasks, which can be seen in the ui, besides grepping the logs?

@Julian for high-level stats, you might want to try a State Handler: https://docs.prefect.io/core/concepts/notifications.html#state-handlers

Thank you!

Of course!