https://prefect.io logo
#prefect-community
Title
# prefect-community
l

Luis Henrique

09/22/2022, 7:33 PM
Hi, guys. Has anyone had any experience on running about 39k flows/day (~100 tasks each on average ~ 4 million tasks daily), and how the database and server scales with this kind of volume?
c

Christopher Boyd

09/22/2022, 9:23 PM
Hi Luis , What is your use case , and are you asking with regards to prefect server or prefect cloud ? That scale amounts to almost 50 api interactions every single second, and that's JUST for task execution. Each flow and task can do a multitude of requests as it transitions through states
l

Luis Henrique

09/26/2022, 6:43 PM
Hi! It would be prefect server self-hosted
Actually it would be for executing several RPAs
c

Christopher Boyd

09/26/2022, 6:49 PM
I think that would be significantly pushing the envelope of what prefect server could do assuming it could handle that - you would need to provision your own DB that would be able to handle that throughput, and self-hosted your only option would be vertically scaling the DB to meet resourcing
I would be very surprised if we have anyone in this community at that scale that is running prefect server