jack
10/07/2025, 5:30 AMMarvin
10/07/2025, 5:30 AMMarvin
10/07/2025, 5:36 AM@flow or using run_deployment creates additional flow runs (separate serverless jobs). Each of those jobs’ wall-clock time is counted.
- Retries: Flow retries create new attempts (new jobs). All attempts count toward usage.
- Heavy startup per job: Large Docker images or installing dependencies at runtime (pip install) can add minutes on every job due to image pull/import time.
- Multiple runs: Schedules, manual triggers, or backfills may have created additional runs in the same window.
- In‑process waits: `sleep`/polling inside the flow keeps the container alive and counts.
What not to sum
- Summing task durations usually won’t match usage. Tasks run inside a single job; parallel tasks don’t multiply usage. The right unit to sum is flow run (and subflow/child) wall‑clock time per serverless job.
How to verify in your workspace
- Filter flow runs by your Managed Work Pool and the time window, then sum the flow run durations (include subflows/child runs). That total should align with usage.
- Open a few runs and compare the job’s start/end timestamps; look for long startup times, retries, or subflow chains.
If you share a couple of run links (parent + any subflows you suspect) and your work pool name, I can help pinpoint exactly where the extra minutes came from.
Ways to reduce managed minutes
- Avoid spawning many subflows/child deployments for short work; keep work as tasks in a single flow run when you can.
- Prebuild and reference a slim Docker image so you aren’t installing dependencies at runtime.
- Be deliberate with retries; make sure failures aren’t causing many re-attempts.
- For longer/heavier testing, consider switching the deployment to a worker-based pool you run (local process, Docker, Kubernetes). Self-hosted workers do not consume managed compute minutes; managed minutes only apply to Managed Work Pools.
References
- Introducing two new self‑serve plans for Prefect Cloud
- Work Pools and Workers: Deploy Python securely
Note: There isn’t a public doc that spells out every metering detail (e.g., rounding), but the practical rule of thumb above matches how usage is tracked in Managed Work Pools.
If you’d like, send me:
- Your workspace slug
- Work pool name
- 1–2 flow run URLs you believe total ~1h30m
I’ll help reconcile them against the usage meter.jack
10/07/2025, 5:40 AMMarvin
10/07/2025, 5:40 AMMarvin
10/07/2025, 5:41 AMdefault-work-pool in your workspace.
Short answer on usage
- Managed Work Pool usage is the wall‑clock time of each serverless job (each flow run attempt) from container start to exit.
- It includes image pull/startup, imports/setup, your flow and tasks running, waits/sleeps, teardown, and each retry or subflow job.
- It does not include time waiting in a queue before the job starts.
Why 1h30m can show ~8.4h
- Subflows or run_deployment calls create additional flow runs (separate jobs) — all their durations add up.
- Retries count as new attempts — all attempts add up.
- Large images or installing deps at runtime adds startup time for every job.
- Multiple runs (schedule/backfills/manual triggers) in the same window add linearly.
- Summing task durations won’t match — usage is per job wall‑clock, not per task.
How to reconcile in your workspace
Option A: In the UI
- Go to the Runs page, filter by Work Pool = default-work-pool and your date range.
- Sort by Start time and review run durations; include any subflows/child runs that occurred in the window.
- Add those flow run durations together — that should align with the usage total.
Option B: Programmatically sum durations
Use the Prefect 3.x client to pull runs for that work pool and sum (end_time - start_time) for each run in your window.
import asyncio
from datetime import datetime, timezone, timedelta
from prefect.client.orchestration import PrefectClient
from prefect.client.schemas.filters import (
FlowRunFilter, FlowRunFilterStartTime,
WorkPoolFilter, WorkPoolFilterName,
)
from prefect.client.schemas.sorting import FlowRunSort
START = datetime(2025, 10, 6, 0, 0, tzinfo=timezone.utc)
END = datetime(2025, 10, 7, 0, 0, tzinfo=timezone.utc)
WORK_POOL_NAME = "default-work-pool"
def wall_clock(run):
if run.start_time and run.end_time:
return run.end_time - run.start_time
# if still running, count until now
if run.start_time:
return datetime.now(timezone.utc) - run.start_time
return timedelta(0)
async def main():
async with PrefectClient() as client:
runs = await client.read_flow_runs(
limit=2000,
sort=FlowRunSort.START_TIME_ASC,
flow_runs=FlowRunFilter(
start_time=FlowRunFilterStartTime(after_=START, before_=END)
),
work_pools=WorkPoolFilter(name=WorkPoolFilterName(any_=[WORK_POOL_NAME])),
)
total = timedelta(0)
for r in runs:
total += wall_clock(r)
print(f"Runs found: {len(runs)}")
print(f"Total managed minutes: {total.total_seconds()/60:.2f} min ({total})")
asyncio.run(main())
If you share:
- the approximate date/time window you’re looking at, and
- 1–2 parent flow run URLs that you believe total ~1h30m,
I can help reconcile them against the usage meter and call out exactly where the extra minutes came from (subflows, retries, startup, additional runs, etc.).
Tips to reduce managed minutes
- Minimize subflows/child deployments for short work; keep logic in one flow run when feasible.
- Prebuild a slim image with dependencies to cut startup time.
- Tune retries so repeated long attempts don’t pile up minutes.
- For heavier/longer testing, consider switching that deployment to a self‑hosted worker pool (local/Docker/Kubernetes) — self‑hosted workers don’t consume Managed Work Pool minutes.
References
- Introducing two new self‑serve plans for Prefect Cloud
- Work Pools and Workers: Deploy Python securely
And here’s the link you shared for context:
default-work-pool