Great question — you can absolutely fan out work so each unit runs in its own Kubernetes Job/pod without creating multiple deployments. High-level pattern - Create a single deployment of your flow to a Kubernetes work pool. - Trigger many flow runs of that one deployment. Each run is picked up by your K8s worker and executed in its own Job/pod. - Pass different parameters per run to partition the work. Option 1: From a parent/orchestrator flow (recommended)

from prefect import flow, get_client
from prefect.deployments import run_deployment
import asyncio
import time

@flow
def orchestrator(batch_size: int = 50):
    # Fan out: create N runs of a single deployment
    run_ids = []
    for i in range(batch_size):
        fr = run_deployment(
            name="worker-flow/worker",           # <FLOW_NAME>/<DEPLOYMENT_NAME>
            parameters={"item_id": i},           # partition your workload
            timeout=0,                           # return immediately; don't wait
            # job_variables can override infra per-run if needed (image, env, etc.)
            # job_variables={"env": {"EXTRA": "1"}}
        )
        run_ids.append(fr.id)
    return run_ids

Notes - One deployment, many runs — each run becomes its own Kubernetes Job/pod. -

timeout=0

makes the call fire-and-forget so you aren’t blocking in the orchestrator. - If you want the runs to be visually attached to the parent flow in the UI, keep the default

as_subflow=True

. Option 2: From any Python process or script

from prefect.deployments import run_deployment

# Trigger 10 parallel runs of the same deployment
for i in range(10):
    run_deployment(
        name="worker-flow/worker",
        parameters={"item_id": i},
        timeout=0
    )

Option 3: From the CLI (simple batching)

# Kick off N runs in the background
for i in {1..10}; do
  prefect deployment run worker-flow/worker -p item_id=$i &
done
wait

When to use which Prefect primitive - Use

run_deployment(...)

to create separate flow runs that each get their own pod. This is what you want. - Calling subflows directly (e.g.,

child_flow(...)

) or

task.submit(...)

keeps execution in the same pod/container as the parent flow — good for in-process concurrency, not for separate pods. Controlling scale and behavior - Scale parallelism by running more K8s workers on your work pool and/or setting concurrency limits in your environment as needed. - Use

job_variables

on

run_deployment(...)

to override per-run infrastructure details (e.g., env vars, image) based on your work pool’s base job template. - Use

tags

or

flow_run_name

for traceability, and

idempotency_key

if you need deduplication when retrying triggers. If you’d like, share: - Your flow/deployment names - How you’re partitioning work (parameters) - Whether you want the orchestrator to wait for all children to finish I can tailor a concrete snippet that fits your setup.