Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hi all! I'm new to AWS, Docker, _and_ orchestration :sweat_smile: I'm starting a data team at a new company and I'm trying to run Prefect as a proof of concept. Right now my plan is to host the server and database on a single EC2 instance inside two containers. The server uses the `prefecthq/prefect:2-latest` image. It's been a pain having to remote into the EC2 instance, then `docker cp` my flow into the server container, then `docker exec` into the container to run `prefect deploy` then copy out that yaml file so I can modify the pull step, and _then_ `docker cp` it back into the server container. It doesn't have ssh or git or I'd just clone the repo that way. Is this what most people do when they self host? If I don't use the yaml file then I have to define my deployment in the code and I'm getting all mixed up. It doesn't feel right to me, so any opinions on what a good workflow for this is would be greatly appreciated!

When I used self-hosted I used fargate for this, and configured github actions to do a lot of this stuff so wanted to just post here to follow and see what others do.

fwiw when I was a prefect user (before I started working here!) I ended up just using cloud since  my use case fit into the free tier (like 400 requests/min at the time or something)

+1 to using Cloud on the free tier if you're doing a POC that doesn't have massive, high-frequency orchestration volume.

Maybe a less dismissive version of my take on this is, if you really want to self-host that's awesome, but if operating a server is getting in the way of actually running workflows, start with Cloud and make running the server the last thing you learn. It's easier to pick up how all the pieces fit together - workers, work pools, deployments, etc. - and enjoy the feeling of stuff working without all the extra effort of hosting. Then you can start up a server wherever you want and the whole process will feel natural.

Smart! I might do that and then try to run them side by side once I get comfortable. Thanks!