Hi, I’m the Data Scientist Leader in a private company that has almost all the services running on AWS. We are planning to create a new data pipeline, but not sure at this point if it is better for us to install prefect in our AWS cloud and then manage it, or go for a Prefect Cloud account. Which are the benefits of Prefect Cloud? What are the computational resources of Prefect Cloud for highly demanding computational intensive data pipelines? The numbers of flow executions are limited by the kind of Prefect Cloud account?

the benefits listed here Oscar from a similar question https://prefect-community.slack.com/archives/CL09KU1K7/p1672763199717319?thread_ts=1672763199.717319&cid=CL09KU1K7

Tx @redsquare ! So, it seems that I didn’t understand very well the concept of Prefect Cloud, since I will have to provide the machines to run the agents, isn’t it? So, we can say that Prefect cloud is like an orchestrator service but not a complete solution to run a data pipeline.

Correct yes

I am in a very similar situation as you are @Óscar Reyes. I personally find that if you can "pay" the engineering price to set the prefect server yourself prefect cloud offers very little advantages compared to the price. It also surprised me that it does not come with agents to run your flow on.

i can speak from my own company’s experience that for us the hybrid model actually works pretty well - since we both like to have a managed solution that hides away the complexities of maintaining and scaling a healthy orchestration system, DB, etc. but - we also enjoy the idea that a single agent running on our own platform (k8s on AWS in this case) makes sure that all our runs happen within our own ecosystem (this is valuable especially in terms of security, privacy of data, and more fine grained control over the properties of our execution environments, resources, etc.) it’s sort of a compromise between two possible extremes (fully managed / vs. fully self-hosted) i also like the idea that there’s support for local agents which would enable our engineers or DS’s to run flows on their own laptops or on a dev-dedicated EC2 machine for example, without having to do anything more complex than simply using some API token of course, this wouldn’t have been the case if we hadn’t already owned a relatively mature K8s cluster with a dedicated DevOps team to manage it

Agree with all that you say @Tom Klein , lovely to have the orchestration, workspaces, security complexity outsourced but keep the flow of our data internal within our own k8s that we already pay for