Does Prefect provide a way to search flow run logs...
# prefect-uim
merlin
10/29/2022, 5:14 PMHi all! I'm testing prefect orion server locally. Is there a way to search flow run logs? I know I can "search by run name" on the flow runs tab, but that is limited utility.
I know the logs exist, I want to search for particular error messages that were logged in the last 100 or so flow runs. I cant see a way in the UI, or even int prefect CLI.
✅ 2
merlin
10/29/2022, 6:20 PMGot it, roughly. Would love to see prefect CLI or python API version of this but.. here is how I got to the UI view of a flow run with an error message logged:
Copy code
$> sqlite3 -readonly ~/.prefect/orion.db
sqlite> select distinct flow_run_id, created from log where message like '%ERROR: Traffic%' order by 2 desc;
46b4e64b-a9cf-4c60-8c58-6766ad0c5ad4|2022-10-29 18:05:53.352675<http://127.0.0.1:4200/flow-run/46b4e64b-a9cf-4c60-8c58-6766ad0c5ad4>gratitude thank you 1
a
Anna Geller
10/29/2022, 6:26 PM
initially, I thought that you are interested in searching for a specific log message content of a given flow run on that flow run's page, but it looks like you want to globally search for that message across flow runs and this is a harder problem and something I would rather expect from a log aggregator rather than from Prefect
are you already using something like Splunk, ES + Kibana or Datadog?
m
merlin
10/29/2022, 6:33 PMnot using those, im just an ersatz DE where i need to support my own work ha ha.
now that you mention it, would love to use a better tool. can you recommend best Datadog usage setup?
my collegues use that
👍 1
a
Anna Geller
10/29/2022, 6:40 PM
not yet, but let me open an issue for tracking, we need to do a better job of forwarding logs to log aggregators -- currently, you would need to do it by attaching custom logger and we can definitely make it easier
btw funny that you used the word Ersatz DE - I'm from 🇩🇪
🇩🇪 1
m
merlin
10/29/2022, 6:45 PMi visited München once in December and loved it.
🙌 1
a
Anna Geller
10/29/2022, 6:50 PM
good choice of timing - all the Christmas markets (Weihnachtsmärkte) make that worthwhile already 🎄
🎯 1
Anna Geller
10/29/2022, 7:03 PM
10 Views