Hello, I'm trying to better understand the pricing/usage model. We have an ETL job that needs to make ~10k API calls to 2 different endpoints so ~20k calls total. We use Prefect's map and LocalDaskExecutor to utilize parallelization. However, if my understanding is correct, a single flow run counts as ~20k task runs and cost either
20,000 * 0.0025 = $50
or
20,000 * 0.005 = $100
depending on the plan you're on?
j
Jenny
04/30/2021, 8:28 PM
Hi @Carter Kwon - the new pricing model is per successful task run. I'm not entirely sure what you mean by "a single flow run counts as -20K task runs" but it sounds like you're asking about a flow that may have 20k mapped tasks?
c
Carter Kwon
04/30/2021, 8:29 PM
You're correct. That single flow has 20k mapped runs tasks. Assuming each mapped task is successful, does running that entire flow once count as 20k successful task runs?
Carter Kwon
04/30/2021, 8:34 PM
I might be getting my terminology mixed up, but here's a screenshot of 1 of the 2 10k mapped runs I'm referring to. There are 2 of these totaling 21,706 mapped runs in the flow.
j
Jenny
04/30/2021, 8:45 PM
I think your terminology is correct. Mapped tasks runs will count as task runs. But (sorry more terminology for you!) for billing we only count task runs that last for longer than one second and that end in a
Success
state.
c
Carter Kwon
04/30/2021, 8:49 PM
Got it. It looks like the majority of these task runs are taking > 1 second so it sounds like we might have to avoid .map for this flow unfortunately. This needs to run daily so that would add up quickly. Thanks for the info.
j
Jenny
04/30/2021, 8:51 PM
No worries - highly recommend the pricing calculator and the Qs section on the pricing page of the website. But let us know if you have any other questions!
c
Carter Kwon
04/30/2021, 8:53 PM
I used that to get my cost calculations. I just wanted to verify I was correct.
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.