Hi team! Im exploring the use of Prefect too. I found the overall product proposition very good. However I was confused with the setup and installation process for AWS ECS (Fargate) while using Prefect Cloud.
As I understand it:
1. Set up Prefect Cloud account ✅
2. Docs say to set up a Prefect Agent? Tried to follow the guide, but was confused about the setup - There is no container image provided for the agent (using AWS Console). How should I go about setting the agent up on Fargate?
3. How many agents does Prefect require? 1 per job? Or is it more like Gitlab/Github runners where you can have a few shared/pooled agents to execute the jobs?
4. Any guidance on the compute size of the agents? I’ll be looking to run frequent 10M record ETLs. So wasnt sure the size of the agents need to support something like that?
Apologies for the simple / dumb questions