Thanks for the detailed breakdown — your analysis is spot on. In Prefect 3.x, that 5–8s gap on AKS is almost entirely the Kubernetes job/pod cold start plus Python startup, not Prefect polling. For on-demand runs, there isn’t a knob in the Kubernetes worker that removes that cost; you need a “warm process” pattern. Answers to your questions: 1) Pattern to reduce startup latency for on-demand runs - Run latency‑sensitive flows in a long‑lived process instead of a per‑run Kubernetes Job. You can do this two ways: - Serve the flow: keep the Python interpreter and imports warm and let it pull runs as they arrive. - Run a Process worker: start a worker in a long‑lived pod and point your low‑latency deployments at a Process work pool; runs start in-process immediately. - If you still need Kubernetes isolation for heavy jobs, split those flows to the Kubernetes worker and keep only latency‑sensitive ones on a Process pool. Example (serve a flow — sub‑second starts once the pod is up):

from prefect import flow

@flow
def fast_flow(x: int = 1):
    return x + 1

if __name__ == "__main__":
    # Starts a long‑lived server that keeps the interpreter hot
    fast_flow.serve(name="fast-flow")

Deploy that container as a Kubernetes Deployment (replicas=N for your target concurrency). Prefect will route runs to those warm processes without creating a new Job per run. Example (Process worker in Kubernetes): - Create a Process work pool in Prefect Cloud (e.g., “process-latency”). - Run a Deployment that starts the worker and keeps it alive:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: process-latency-worker
spec:
  replicas: 2
  selector:
    matchLabels: { app: process-latency-worker }
  template:
    metadata:
      labels: { app: process-latency-worker }
    spec:
      containers:
      - name: worker
        image: your-registry/your-image:tag  # pre-bake Prefect and your code
        env:
        - name: PREFECT_API_URL
          value: <https://api.prefect.cloud/api/accounts/<acct>/workspaces/<ws>>
        - name: PREFECT_API_KEY
          valueFrom:
            secretKeyRef:
              name: prefect-api-key
              key: token
        - name: PREFECT_WORKER_QUERY_SECONDS
          value: "2"  # optional, faster pull for on-demand
        command: ["prefect", "worker", "start", "-p", "process-latency"]

Then point your latency‑sensitive deployments at the “process-latency” pool:

flow.from_source("<https://github.com/org/repo>", entrypoint="flows/fast.py:fast_flow").deploy(
    name="fast-flow",
    work_pool_name="process-latency"
)

Result: on-demand runs start almost instantly because there’s no Job or container cold start. 2) Warm pod pool / standby job - There isn’t a native “warm pod pool” or “standby job” feature in the Kubernetes worker today. Jobs are intentionally ephemeral and exit after each run. - The supported way to keep things warm is to run a long‑lived Deployment (either via

flow.serve(...)

or a Process worker) and scale replicas for your desired concurrency. - If you must remain on Kubernetes Jobs, the only mitigations are cluster-level: keep a small node pool always on, pre‑pull images, use

imagePullPolicy: IfNotPresent

, minimize admission webhook latency, and pre-bake as much code as possible into the image. These help, but they won’t remove the job/pod startup cost. 3) Hybrid setup - Yes — this is a common and recommended pattern: - Kubernetes worker + K8s work pool for scheduled/heavy/isolated workloads. - Process worker or served flows (possibly running in Kubernetes as a Deployment) for latency-sensitive on-demand traffic. - Route deployments by work pool. Keep images for the process path lean and pre-baked to minimize import overhead. 4) Upcoming Prefect features

- I’m not aware of a native “warm Kubernetes job” or job-reuse feature on the near-term roadmap. Prefect’s existing lever for scheduled flows is

prefetch_seconds

, which you’ve already used; for on-demand runs, the recommended approach is long‑lived processes (serve or Process workers). - If a first‑class “warm runner pool” for the Kubernetes worker would help your use case, I’m happy to file or upvote a feature request so we can track it. Extra, if you stay on Kubernetes jobs: - Pre-bake Prefect + your flow code + dependencies into the image (so “deployment pull step” does little or nothing). - Keep a small “always on” node pool for this workload; use node selectors/affinity so these pods don’t land on scale‑to‑zero pools. - Set

imagePullPolicy: IfNotPresent

in your base job template and ensure images are cached on those nodes. - Avoid initContainers or heavy startup scripts; keep the entrypoint as lean as possible. Open questions to tailor the recommendation: - Are you open to running one or more long‑lived pods (Deployment) for the low‑latency flows? - Do those flows require strict per-run isolation (e.g., resource limits or secrets) that would force Kubernetes Jobs? - Roughly what p95 startup target are you aiming for (<1s, ~2–3s, etc.) and what burst concurrency do you need? If you can run a small always-on Deployment for these flows, using

flow.serve(...)

or a Process worker is the most direct, “native” Prefect way to get sub‑second start times for on-demand triggers.