Hi, I have a pipeline in AWS where things trigger each other and would like to move it entirely to prefect but have some doubts/questions:
1. There is a Lambda_1 triggered on a schedule via cron job in Eventbridge. At the end this Lambda_1 has some python code to trigger Lambda_2
a. Q1.1) Can I trigger Lambda_1 in a Prefect deployment?
b. Q1.2) Can Lamba_2 be triggered as well from Prefect depending on the completion status of Lambda_1?
2. There is a Lambda_2 that at the end of its python code does the following: updates the Fargate service for Container_1 to desired count = 1 (basically starts a task that wil be billed starting then).
a. Q2.1) Can I update, with Prefect, the service for Container_1 to have 1 task running, depending on the completion status of Lambda_2?
b. Q2.2) Can I start, with Prefect, the Prefect deployment for Container_1 depending on the completion status of the service update?
c. Q2.3) Can I, with Prefect, update the service again to 0 so that it stops the billing? this should also depend on the completion status. If all python code was succesfull then stop and if not then retry a couple of times the flow for the deployment in Container_1
3. The Container_1 has a Prefect deployment that at the end calls Lambda_3 and Lambda_4
4. Lambda_3 is similar to Lambda_4 and it starts updates another fargate service to have 1 task running. Then it should start the Prefect deployment inside Container_2
5. Container_2 is similar to Container_1. It should call another lambda if the python code run succesfully and should retry a couple of times if there was any errors.
At the moment I have all these triggers/calls within the codes and have to no proper way to see unified logs for everything so I thought Prefect could help and simplify my experience.
General suggestions on the overall architecture are also welcome :)Thanks!