I am new to Prefect and want to understand couple of things.
• I want to run flows in GPU instance. This means that I have to run the prefect agent in GPU. Does this instance needs to be switched on all the time? How can we scale down/up based on the requests?
• I want some other system to trigger the flows (backend systems in my org). For that I am thinking to write APIs for all the flows and trigger them when an API is called. If flows are invoked like this, will this still be shown in the UI?
• How exactly prefect agent knows what code to run? Does prefect core code needs to be running all the time?
• What are some best practices used for building applications using prefect? - I am getting confused regarding how code is managed
Thank you for the help provided 🤗
04/27/2022, 9:46 AM
#1 Regarding your first question, I wrote about exactly the same topic here
#2 Your backend system can trigger a flow run via an API call - you don't even have to install Prefect to do that, just using e.g. requests as shown here will suffice
#3 It knows it through labels - you attach a label to your flow run configs and the same label to your agent, this way they get matched. The agent needs to be running 24/7 to constantly poll for new flow runs
#4 Some best practices are listed here but ask if you have any specific questions. The code is managed via the Storage abstraction and a Hybrid Execution Model