Hi, guys. Has anyone had any experience on running...
# prefect-community
l
Hi, guys. Has anyone had any experience on running about 39k flows/day (~100 tasks each on average ~ 4 million tasks daily), and how the database and server scales with this kind of volume?
c
Hi Luis , What is your use case , and are you asking with regards to prefect server or prefect cloud ? That scale amounts to almost 50 api interactions every single second, and that's JUST for task execution. Each flow and task can do a multitude of requests as it transitions through states
l
Hi! It would be prefect server self-hosted
Actually it would be for executing several RPAs
c
I think that would be significantly pushing the envelope of what prefect server could do assuming it could handle that - you would need to provision your own DB that would be able to handle that throughput, and self-hosted your only option would be vertically scaling the DB to meet resourcing
I would be very surprised if we have anyone in this community at that scale that is running prefect server