Hey has anyone ran the open source server at scale...
# ask-communityp
Patrick Barker
10/05/2023, 8:38 PMHey has anyone ran the open source server at scale? Trying to understand if it can make it to 10k plus workloads
d
Deceivious
10/12/2023, 12:42 PMWhats a 10k plus workload?
Deceivious
10/12/2023, 12:42 PMp
Patrick Barker
10/18/2023, 6:00 PMSweet! How many simultaneously have you ran?
d
Deceivious
10/18/2023, 6:05 PMAnywhere from 5 to 45 ish
Deceivious
10/18/2023, 6:06 PMI could query the database and find the exact count
i
Idan Noti
12/08/2023, 3:33 PMHi @Patrick Barker Did you get to run prefect at scale with 10K workloads?
d
Deceivious
12/08/2023, 4:12 PMWhat does workload mean in this context
p
Patrick Barker
12/08/2023, 5:32 PMWe implemented it but haven't load tested it yet
i
Idan Noti
12/09/2023, 10:28 AMThanks, we also experience some problems making prefect OSS run at scale. We have 100s of flows running at the same time and it seems that our DB starts to responds slowly when the load becomes larger.
@Deceivious I also saw you question about something in this context, did you manage to have a working solution in a heavy load such as this?
d
Deceivious
12/19/2023, 9:17 AMPrefect OSS makes a lot of request to the database. I dont think there is an easy way around this tbh.
Deceivious
12/19/2023, 9:18 AMI think this falls more into optimizing your database stack.
p
Patrick Barker
12/19/2023, 3:55 PMI talked to another dev that said a similar thing, the key is scaling the DB