Hi all, I've started using prefect and I think its a great api for managing workflows (I see it as a human friendly wrapper around dask orchestration). Thanks for excellent documentation and a super useful tool! I'm trying to determine how best to use prefect within an orchestration workflow design. The workflow is essentially three generic steps: 1. spinning up various services (various web servers and processes) 2. manage these services: state, variables/properties, tasks (prefect seems to fit this, although I may need to find a good way to store state/data) 3. detect failure/errors in services and trigger redundancy/recovery tasks I am torn on how to use prefect with 1 and 3. I don't think using prefect alone is the best solution, but I also think prefect(dask scheduler/executor) is a good solution for starting services (1) and handling triggers (3). However I'm not sure if prefect is ideal for active monitoring. Is there a prefect usecase for monitoring log files/services, or if there is a better tool for this job? I didn't see any mention of monitoring within the prefect core. I'm not talking about monitoring specific prefect tasks which is obvious that prefect should monitor its own tasks, more so on monitoring errors/issues in services. If prefect isn't ideal for active monitoring of services, I envision using something like prometheus to watch data/logs from the services and trigger redundancy/recovery tasks in prefect if prometheus alerts or detects failures/issues. Are there any ideas for a better integration of prefect into this workflow?

Hi @Michael Reeves! What you’re describing is possible with Prefect. We don’t have established patterns for active monitoring, but you could run a flow on an interval schedule to checks things. However, I think a better pattern would be for prometheus to kick off recovery Prefect flows using Cloud or Server. We’d love to know how you get things set up!

@Michael Reeves are you THE michael reeves from youtube?

@Dylan thanks for the answers :D Excellent point for 1, and that helps clarify some of my concerns there. For 3, I think thats an excellent point on having prometheus spin up prefect flows based on alerts! Thank you for the advice 🙂 I will let you know how setting up this goes 😄 if I can (due to work constraints), I may publish how I setup everything! @Zach while I wish I was, I am not the same haha

Great! Let us know if we can help 😄