https://prefect.io logo
r

Rafał Bielicki

01/31/2023, 9:20 PM
Hello everyone! I implemented prefect open source in my company and we love it. I have a couple of questions though. We have pretty basic setup with k8s + RDS Postgres. Our flows have pretty big parameters (JSON with around 50k keys), so as you can imagine our flow_run parameters don’t fit in block size of postgres and so gets toasted. Not sure if toasting is the reason for the issue, but Prefect UI is loading like 20 seconds. I verified network connection between RDS & Orion (they are in same region), I checked EXAPLIN ANALYZE on query that orion executes to get flow_runs, and it blazing flast but when the query is executed it loads really long. Seem like client write issue, meaning client is taking so long to process received data from RDS. Did you encounter something like this? Or do you have any ideas how I can tackle this issue?