Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

image.png

Hi, Prefect stores all these information about flow run and shows nicely with UI like below, is the a way to download run data so we can perform analysis. Eg, average duration of task..etc

you can query logs by flow run: <https://docs.prefect.io/orchestration/flow-runs/inspection.html#getting-flow-run-states>

that's the only way i've found to do it, short of sending all logs somewhere else at runtime

For task duration, you can use the GraphQL api to get that start time and end time of a task, but you need to process it further with Python to get duration since we don’t have duration calculated in the database

I am thinking of ingesting these data to our database. We are using Standard Plan, so these data is kept for 14 days?

Yes that’s right. So other people pump their logs to a logging tool like DataDog or something