What's the recommended way to add monitoring and observability to a prefect flow? I mean:
• the ability to see input-output for tasks (at least for a while after running them; just not Results)
• the ability to see timings for each task
• any structured logging from inside tasks (e.g. especially important segment variables / which app / tenant is it running for)
• integrating with Prometheus
?
a
Anna Geller
10/22/2021, 11:08 AM
@haf I believe the UI is able to answer most of those questions. The TaskRun page shows the duration, and other metadata such as where the Result of the task is stored, state of this task, and the upstream task, and the logs of this specific task.
Anna Geller
10/22/2021, 11:11 AM
Prometheus would need to be configured. Forwarding logs to an external log aggregation system is very much dependent on the infrastructure you are running on. I remember you run on Kubernetes on GCP (correct?). Perhaps this GCP documentation can help to set this up?