https://prefect.io logo
Title
a

ale

03/11/2022, 10:36 AM
Hey folks 👋 We’ve been using Prefect Server for two years now and are really happy with its flexibility! However, keeping Prefect Server up to date is becoming quite expensive (1 Data Engineer + 1 SRE need to spend 1 week every time we want to upgrade). So, we are evaluating the ability to switch to Prefect Cloud. Given that pricing is based on successful task runs, I have few questions: • what’s the best way to get daily task run stats out from Prefect Server? • how does task/flow mapping fit into Prefect Cloud pricing? Regarding the first point, I crafted up a query but I’m not sure it is correct. Any help would be much appreciated! 🙌
with raw_data as
(
	select "start_time"::date as task_run_date,
	       state,
	       count(id) as task_run_per_date_and_state
	from public.task_run
	group by 1,2
)
select state,
       avg(task_run_per_date_and_state)::int as avg_task_runs_per_day
from raw_data
where state = 'Success'
group by 1
order by 1
a

Anna Geller

03/11/2022, 10:56 AM
Thanks so much for sharing and I totally get that self-hosting things is always challenging. To asses the number of billable task runs you could normally run this query:
{
  billing_usage(order_by: {timestamp: desc}) {
    runs
    timestamp
  }
}
But just tested this and noticed this is not available on Server. So you would need to do:
query {
  task_run_state_aggregate(where: {
      timestamp: { _gte: "2022-03-01T00:00:00.000000+00:00" }
      state: { _in: ["Success"] }
    }) {
    aggregate {
      count
    }
  }
}
Then, you would need to exclude the tasks that took less than a second. Regarding the SQL query you mentioned, I believe @Dylan had some SQL query for that purpose (I can recall I saw it in some thread but Slack deleted it by now). Mapped child tasks are independent fully-featured Prefect tasks and they count for billing the same way as any task, i.e. they are billed only if they are successful and take longer than a second. Also note that we never bill for retries and task run failures in general.
a

ale

03/11/2022, 11:44 AM
Thanks a lot @Anna Geller 🙌
👍 1