Great question — and thanks for the detailed context. Short answer - What you’re seeing (160–200Mi per flow-run pod) is normal for Python + Prefect in Kubernetes. Each run is a full Python interpreter plus the Prefect engine and deps. - Prefect is a good fit for your use case, but you won’t want 100k pods at once. The trick is to: - limit concurrent runs at the work pool level, and/or - model your workload to run many lightweight HTTP calls inside a single (or a few) flow runs using concurrency. Why each pod uses ~200Mi - A K8s work pool launches a fresh pod per flow run. That pod has a Python process, Prefect engine, http client, logging, etc. A ~160–200Mi RSS baseline is typical for that stack on python:3.11-slim. How to scale this kind of workload efficiently 1) Limit concurrent pods (work pool concurrency) Only allow as many simultaneous runs as your cluster can handle. Prefect will queue the rest and execute them as capacity frees up. - Set a work pool concurrency limit:

prefect work-pool set-concurrency-limit "YOUR-K8S-POOL" 50

Docs: CLI: work-pool, Kubernetes work pools 2) Run many HTTP calls inside one flow run Instead of 1 flow run = 1 HTTP call, use a single flow run (or a small number of runs) to fan out many calls concurrently. This keeps memory to a few hundred Mi per run while driving very high throughput. - Threaded example using a task runner and streaming results:

import httpx
from prefect import flow, task
from prefect.futures import as_completed
from prefect.task_runners import ThreadPoolTaskRunner

@task
def call_endpoint(url: str, timeout: float = 10.0) -> int:
    with httpx.Client(timeout=timeout) as client:
        r = client.get(url)
        return r.status_code

@flow(task_runner=ThreadPoolTaskRunner(max_workers=200))
def bulk_http(urls: list[str]) -> list[int]:
    futures = [call_endpoint.submit(u) for u in urls]
    results = []
    for fut in as_completed(futures):
        results.append(fut.result())
    return results

- Async example (efficient for I/O-bound calls) with concurrency caps:

import asyncio
import httpx
from prefect import flow, task

@task
async def call_endpoint_async(client: httpx.AsyncClient, url: str) -> int:
    r = await client.get(url, timeout=10.0)
    return r.status_code

@flow
async def bulk_http_async(urls: list[str], concurrency: int = 200) -> list[int]:
    sem = asyncio.Semaphore(concurrency)
    limits = httpx.Limits(max_connections=concurrency)
    async with httpx.AsyncClient(limits=limits) as client:
        async def bounded(u):
            async with sem:
                return await call_endpoint_async(client, u)
        return await asyncio.gather(*(bounded(u) for u in urls))

3) Apply global concurrency limits to protect external systems If multiple flows/deployments call the same API, use Prefect’s concurrency limits by tag to enforce a global cap across your entire workspace. - Tag your task:

@task(tags=["external-api"])
def call_endpoint(...):
    ...

- Create a global concurrency limit:

prefect concurrency-limit create external-api 200

Docs: Global concurrency limits 4) Consider a Process worker in Kubernetes for short jobs If your flows are extremely short-lived, running a long-lived Process worker as a single Deployment in your cluster (instead of spinning a new K8s pod per flow) can reduce Kubernetes pod churn. Each run still uses a Python process, but you avoid the overhead of creating/destroying pods thousands of times. Docs: Workers and work pools 5) Right-size K8s resources in your job template

For Kubernetes work pools, set your pod requests/limits appropriately in the base job template (e.g., request 100–200Mi, limit 256–512Mi) and validate via load testing. Prefect’s K8s guide shows where these fields live in the job template. Docs: Kubernetes work pools Notes on your Dockerfile/CMD - Your CMD uses

-m

with a path containing slashes. Python expects a module path with dots when using

-m

. If it’s working as-is, great — otherwise consider:

CMD ["python", "-m", "src.flows.${FLOW_NAME}.main"]

or

CMD ["python", "src/flows/${FLOW_NAME}/main.py"]

Where I’d start - Decide the maximum concurrent HTTP calls you actually need (e.g., 200–1000), not how many total you need to run. - Implement the single-run fan-out pattern (threaded or async). - Set a work pool concurrency limit so you never launch more pods than your cluster can handle. - Optionally add a global concurrency limit tag if multiple deployments hit the same API. A few verified CLI references - Set work pool concurrency:

prefect work-pool set-concurrency-limit "my-pool" 50

- Create a concurrency limit on a tag:

prefect concurrency-limit create external-api 200

- Start a worker (if you go the Process worker route):

prefect worker start --pool "my-process-pool"

Questions to help tailor this further - Do you need all 100k calls to be simultaneous? What’s your target QPS and acceptable end-to-end latency? - Are you open to batching the calls into a small number of high-throughput flow runs? - Roughly how big are the responses and how long does each request take? Links for deeper dives - Work pools and workers: Concepts: Workers - Kubernetes work pools and job templates: Kubernetes how-to - CLI reference for work pools: CLI: work-pool - Global concurrency limits: Global concurrency limits - Scaling patterns with mapping/concurrency (background): Prefect blog: mapping and scale Happy to sketch a concrete example with your target concurrency and endpoints if you can share those details.