Howdy everyone, I’m Sofian, I’m working as Site R...
# introductions
s
Howdy everyone, I’m Sofian, I’m working as Site Reliability Engineer in a cloud computing company in France (OVHcloud). We are using Prefect to create business workflows in order to automate several aspects of our automation for Infrastructure Resiliency. From async job to move data from cluster to cluster, to self-repair and replacement of persistent medium of storage (HDD, SSD, NVME, etc.) to simple automation on-demand. We have lot of steps to perform with different internals APIs to build these flows (we called them robots). We started with Prefect 1.0, then migrated to Prefect 2.0 and now running self-hosted Prefect Clusters Service in
2.9.0
version. So far we run more than 100k flows and tasks and we monitor through Grafana our SLOs regarding tasks/flows failures. It used as a massive Scheduler for us and we push to use it even more nowadays for diverse data operations. Some figures as today: • Started to implement Prefect 1.0 in April 2022 • Migrated to Prefect 2.0 in Nov 2022 and did several versions upgrade (~10) up to current version
2.9.0
• Used it for our internals workflows and as backend scheduler • Total Flow runs 105 997
reported by the DB with SELECT count(*) FROM flow_run;
• Total Task runs 272 308
reported by the DB with SELECT count(*) FROM task_run;
15 deployment and 10 different work-queue • 4 Clusters • Some Failures 🙂 and even more of Success 👏 We adopted it because it’s Python based and the learning curve is very slight. Also the dynamics around the project are extremely positives. Happy Positive Engineering 🙂 Just wanted to give you a feedback and Thanks for this workflow framework 🤝
💯 13
👋 7
marvin duck 3
🔥 15
d
Hey Sofian! Thanks for the kind words, pleasure to have you here!
👍 2
c
Thank you so much for this wonderful post and the stats Sofian! Great to hear why you use Prefect.
👍 2
b
We're grateful to have you in the community @sbrabez!
👍 2
j
Cool, @sbrabez! 🎉
👍 2
b
Happy Positive Engineering! Love to hear it, thank you @sbrabez 🚀
👍 1
j
So glad to have you here @sbrabez, welcome! 👋
👍 1
a
Hey Sofian! Thank you for the feedback and the really cool stats + congrats on getting over to 2.0! 🚀
👍 1