https://prefect.io logo
Title
s

ssingh

03/21/2023, 8:13 PM
Hi all, Just getting started with Prefect, with some limited exposure to Airflow. Prefect looks promising but I could use your input for the deployment question below. To keep the cost low, I want to run my orchestration tool with either of the two options: a) self managed using AWS EC2 b) managed EKS on AWS Airflow can be deployed in both these ways. Can I deploy Prefect 2.0 (open source) using both/any one of these options? I read the docs(https://docs.prefect.io/concepts/deployments/) and some blogs on how people deployed Prefect but I am still confused. Again, I want to use the open source version of Prefect 2.0 and deploy it using Kubernetes on self managed AWS EC2 or AWS EKS. Thanks for your feedback.
1
r

redsquare

03/21/2023, 9:59 PM
both are possible, I find eks slow as it does not cache images and variable perf so would look to ecs
🙌 1
s

ssingh

03/22/2023, 1:31 PM
@redsquare Could you link some blogs where they show the steps of how to use Prefect 2.0 open source on AWS, and possibly how to scale it? I was following blogs like 1. https://medium.com/the-prefect-blog/seamless-move-from-local-to-aws-kubernetes-cluster-with-prefect-f263a4573c56 2. https://discourse.prefect.io/t/aws-how-to-deploy-a-prefect-2-0-agent-to-an-eks-kubernetes-cluster-on-aws-and-connect-it-to-cloud-2-0-backend/1046 3. https://discourse.prefect.io/t/how-to-deploy-a-prefect-2-0-agent-to-an-ec2-instance-as-your-execution-layer/551 But they talk about Prefect 2.0 cloud. I don’t want to use Prefect 2.0 cloud because it is paid. I want to use the open source version.
r

redsquare

03/22/2023, 1:37 PM
@ssingh I use the cloud so can only reference experience with agents+flows - no experience hosting/scaling/ha'ing the server aspect
n

Nate

03/22/2023, 5:55 PM
hi @ssingh
I don’t want to use Prefect 2.0 cloud because it is paid.
just fyi, you can use free tier prefect cloud if you want, that way you don't need to run a server/API and you can focus on deploying your workflows on EKS / ECS (even if only during your PoC) if you definitely need to host your own server, I'd recommend using the helm chart for a k8s deployment somewhere like EKS
🙌 1