Hi everyone! My name is Taras, and I’m a data scientist. I’ve decided to take a closer look at Prefect for managing ML model’s training and backtesting pipelines. I used Luigi and DVC in the past and am keen to explore python-native dynamic orchestration now 🙂 I’m also interested in running flows in the cloud, so I will probably ask questions about infrastructure.

Hi @Taras Svirskyi! Welcome to Prefect!

Hi, welcome!

Welcome to the community @Taras Svirskyi! DVC is interesting, were you using it in combination with Luigi or standalone?

Hey, welcome @Taras Svirskyi! 👋 Joined us just in time for Meme Friday over in #C01TK7XSLT0 😀

I’m using DVC standalone. Luigi was my first attempt to manage ML code but I end up writing too many boilerplate in attempt to support dynamic pipelines. The DVC provides nice abstractions for metrics/parameters/steps of the pipelines + cache. Yet it becomes quite hard to manage all steps as pipelines grow cause it’s yaml+python.

Welcome aboard, @Taras Svirskyi! Great to have you with us!