Hi guys, below is question about “the prefect product” logging.
I can’t slice and dice logs the way I am used to with the prefect-ui interface.
So I wonder product wise, if prefect’s logging ui is a simple interface just meant to easily search the logs grouped by task or flow, and it should be used side by side with a more powerful logging solutions ?
More powerful logging solutions such as elasticsearch+filebeat or even just saving them in a file and going over them with
lnav
k
Kevin Kho
01/04/2022, 5:40 PM
A lot of our users combine Prefect with other logging tools like DataDog. Considering we only keep logs for a 2 or 3 week period also for non-enterprise accounts, you probably get more from other tools for logging.
That said though, you can try pulling the logs from the GraphQL API and manipulate it with Python if you need to
👍 1
k
Kevin Mullins
01/04/2022, 6:40 PM
@Kevin Kho I’m interested in getting the logs to DataDog. Do you happen to know if there is any information/patterns available around this?
Just trying to figure out what a good approach would be.
k
Kevin Kho
01/04/2022, 6:41 PM
Check this discussion and maybe chime in. Billy is friendly
👍 1
k
Kevin Mullins
01/04/2022, 6:42 PM
Awesome, thanks for pointing me in the right direction!
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.