Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

We are running prefect for 90 days in production. Today we noticed that the Prefect Server is querying Postgresql at a rate of 14MB/s. Grew from 3MB/s linearly over a course of a month.
Is this something you are aware of?
We nuked the db now and checking if this is related to the number of flows we ran so far.

image.png

The query insights look a bit crazy. Many queries running 30.000 times an hour
10 times a second.

Our empty prefect (few blocks, 2 flows, 10 runs altogether) queries the database at 100-200 kB/s.
Is this normal?

This morning after some flow runs, it is querying the database at 0.25MB/s. Megabyte!

Is this normal?

I am enabling the query insights on my server as well. We have been running the server for over 6 months now. Will check to see if I find something interesting.

we have been running for an year now.. ill monitor it as well :open_mouth:

<@U03RN8W7DPU> <@U042TC9FVR8> did you see anything on your end?

Hey <@U05ENMYMK2Q> as your database gets larger and more things accrue I would definitely expect this number to go up generally.

However, I do think this number seems pretty high, especially related to what your workload appears to be? Would you mind filing an issue <https://github.com/PrefectHQ/prefect/issues/new?assignees=&labels=needs%3Atriage%2Cbug&projects=&template=1_general_bug_report.yaml|here>?  Any/all information you would be willing to share would be helpful, your setup, state of your db, type of workloads, flow param size etc.

Thank you. We may found one candidate for the root cause.

Our task triggers carry a payload of 500kb json. Trying to reduce that and measure again.