Hi there. We want to run a small instance of Prefe...
# prefect-serverp
Pedro Machado
12/12/2020, 12:24 AMHi there. We want to run a small instance of Prefect in an AWS instance.
We'll probably be using the Docker agent to run some fuzzy matching (entity resolution) flows in the same instance.
This is a beta of a service we may grow, but in the beginning we wanted to start with simple infrastructure. Unfortunately, Prefect Cloud is disallowed at this stage because it would have to be vetted by our client's security team and that can't happen in the near future and that's why we are starting with Prefect Server.
My question is ... do you have any recommendations as to which instance type and size we might want to start with to run Prefect Server? Some flows will be somewhat memory and CPU intensive but we won't be running a high volume of them. Thanks!
z
Zanie
12/12/2020, 2:26 PM
Hi Pedro! I’ll ask our dev ops team about this on Monday. Until then maybe someone else from the community will have recommendations 🙂
p
Pedro Machado
12/16/2020, 8:13 AMHi Michael. Did you get any recommendations from your devops team?
z
Zanie
12/16/2020, 4:47 PM
I asked and did not get a reply 🙂 I’ll ping them again
Zanie
12/16/2020, 6:36 PM
There are a ton of variables here (mapping, logs, etc) so it’s hard to give a good recommendation. I’d say for just running Server, you could start relatively small. Since you are intending to run your flows on the same instance, that will likely drive the requirements. For example, if your flows are memory intensive (even occasionally), you will need to have the available memory all the time.
p
Pedro Machado
12/16/2020, 9:03 PMThanks
Pedro Machado
12/16/2020, 9:04 PMWe are starting with a bigger machine and scale down as we understand the workload better.
2 Views