Hi folks, I lead an ML and data science team, and I’m evaluating a bunch of tools in this space. Prefect looks very nice indeed, but I am having trouble understanding exactly what I get with a paid plan vs the open source core. I have looked at all the feature lists I can find, but I guess I’m missing one of those classic “feature comparison tables” that one often sees. Can someone point me in the direction of something like that?

Hi @Christopher, next week we’re open-sourcing our server and UI which will blur the lines a bit from today, but gives us a good opportunity to re-clarify. In a nutshell: paid features include managed infrastructure (including auth, scale / HA, and security layers) as well as team management and permissions. Additional protective measures like services that enforce SLAs, attempt to restart flows that have stopped for unusual reasons are available. Platform also includes features that require global coordination of state, like concurrency limits for task, and third party integrations like Slack notifications.

This belongs in #CKNSX5WG3

You know you’re right

@Christopher We use Prefect Cloud (and used Core prior to that) for our data science workflows to train lots of ML models on healthcare data. If I can help answer any questions, feel free to DM me or just ask here. (Also wrote a high-level blog post about it late last year: https://www.symphonyrm.com/galaxy-brain-data-science-workflows-with-prefect-dask/ )

Thank you for the responses. 1. That announcement looks like great news 2. Thanks @Joe Schmid for the blog. I will read it and DM you if I have questions.

Hi @Christopher - don’t worry, it’s a natural comparison. In fact, I’m actually an Airflow PMC member. Prefect was explicitly designed to solve the issues with using Airflow in a modern PyData stack (or modern stack period, to be honest) — here’s our own take: https://medium.com/the-prefect-blog/why-not-airflow-4cfa423299c4 Typically, the difference between Prefect and most other workflow managers is fundamentally philosophical. For example, Prefect’s goal is to be workflow-agnostic and infrastructure-agnostic. That means our focus is on expressing a really powerful but lightweight syntax for defining workflows, and our challenge is making sure that can be applied in extremely general environments. We want Prefect to become the universal “glue” for whatever tools meet your needs. In contrast, a system like Kubeflow is much more opinionated: it’s for deploying machine learning on Kubernetes. If that’s your only purpose, then Kubeflow may have a more immediate impact, since your work already conforms to its assumptions. If that’s not how you already work, then you’re going to struggle with Kubeflow. At the end of the day, you should use the best tool for the job. If a more opinionated tool meets your needs, by all means use it! But if you’re experimenting, or you like having a local experience that’s identical to your deployed one, or you work in multiple environments or for multiple stakeholders, you may prefer Prefect’s more general Swiss-army-knife approach. We try to make it as easy as possible for you to dive in and find out!

Thanks Jeremiah, really appreciate the reply. That gels with my own reading, so I feel like I have enough of a grip to get started on Monday. Really appreciate the engagement, thanks 😊

Anytime! Hope you find Prefect useful 🙂