Hey all, I’ve been looking into Prefect as a potential tool for a streaming data pipeline, and I feel like I might be misunderstanding something about the pricing model, so I’m wondering if someone can help me sort it out.
After reading [this blog post](https://www.prefect.io/blog/you-no-longer-need-two-separate-systems-for-batch-processing-and-streaming/), I was pretty excited about trying Prefect for this use case. When I looked at the pricing page, I saw that the unit being used to measure usage was task runs. So I guess I have two questions about that:
1. Is it fair to say that an implication of paying for task runs is that one would want to minimize the number of tasks in their flow, so as to get the most “bang for their buck?” This seems counter-intuitive to me; I’m on Page 3 of the Prefect Docs, “Thinking Prefectly”, and one thing I’m pretty sure I know by now is that smaller and more discrete tasks are better. Plus, that blog post (rightfully) pointed out that a Flow is better orchestrated by breaking up the logic into multiple tasks as needed.
2. Is it also fair to say that given that a streaming data pipeline pulls events from a stream at some chosen interval, and presumably each pull constitutes at least one “task run”, the cost of the Prefect job is proportional to the rate at which data is processed? For example, is running a Flow which is efficiently able to pull and process batches of messages every 2 seconds going to be twice as expensive as a Flow which is pulling and processing new batches every 4 seconds? (by the way, when I plug the number of monthly task runs equivalent to continuously pulling batches of messages every 2 seconds into the Pricing calcuator, it comes out to… Contact Us 😄)
TIL Slack does not support Markdown link formatting. Or maybe I just did it wrong.
… and maybe this use case is a level of throughput for which I should indeed “Contact Us”, but I wanted to make sure I wasn’t doing anything stupid first
05/15/2022, 11:54 PM
To alleviate your concerns, Pricing for Cloud 2.0 will be different and it will no longer be purely based on task runs in order to accommodate use cases such as yours. We'll keep you posted about that in the #announcements and via Discourse
05/15/2022, 11:59 PM
Interesting! Are you able to share a rough timeline for that? Are we looking at weeks or a few months?
05/16/2022, 12:05 AM
Are you asking about when self-served Cloud 2.0 pricing will be publicly announced?
I also want to make sure you know that you can currently sign up for Cloud 2.0 beta and use it entirely for free https://beta.prefect.io/
05/16/2022, 12:16 AM
I suppose I am, yeah. Does “self-served” here mean rolling my own deployment?
Oh wow, well that’s also good to know
05/16/2022, 12:20 AM
By that I mean not an Enterprise contract - you can sign up yourself without having to talk to anyone from Sales
@Nash Taylor I can ask someone from Sales to reach out to discuss pricing with you. You can DM me your email address if you're interested
05/16/2022, 12:33 AM
That would be great actually. Also thanks for the quick response on a Sunday!