A Kubernetes worker getting

OOMKilled

immediately usually means the worker pod itself is running out of memory during startup — not necessarily the flow-run job pod. A few likely causes in Prefect 3.x: 1. The Kubernetes observer is using too much memory on startup - Prefect Kubernetes workers can start an observer that watches pods/jobs - In larger clusters, that startup can be memory-heavy - This is one of the most common causes of immediate worker OOMs 2. The worker pod memory limit is too low - If the worker container has a small limit like

128Mi

or

256Mi

, startup overhead alone may exceed it - This is especially common in Helm deployments with restrictive defaults 3. Large/custom image or heavy imports in the worker image - If your worker image installs lots of dependencies, startup imports can increase memory usage - Less common than #1, but still possible 4. Cluster/node memory pressure - Sometimes the pod is scheduled onto a node with very little available memory, making OOM more likely Important distinction - If the worker pod is OOMKilled: the worker dies before or while polling/submitting jobs - If the flow-run job pod is OOMKilled: the worker stays healthy, but individual flow runs crash What I’d try first 1. Increase worker memory If you're using Helm or a pod spec, bump the worker resources. For example:

resources:
  requests:
    memory: "512Mi"
    cpu: "250m"
  limits:
    memory: "1Gi"
    cpu: "500m"

2. Disable the Kubernetes observer temporarily This is the fastest test:

PREFECT_INTEGRATIONS_KUBERNETES_OBSERVER_ENABLED=false

If the worker stops crashing after that, the observer is the likely cause. 3. Reduce observer startup load If you need the observer, try lowering startup concurrency:

PREFECT_INTEGRATIONS_KUBERNETES_OBSERVER_STARTUP_EVENT_CONCURRENCY=1

You can also scope it to fewer namespaces:

PREFECT_INTEGRATIONS_KUBERNETES_OBSERVER_NAMESPACES=prefect

4. Check whether it’s truly the worker pod Run:

kubectl describe pod <worker-pod> -n <namespace>
kubectl logs <worker-pod> -n <namespace> --previous
kubectl top pod <worker-pod> -n <namespace>

Look for: -

Reason: OOMKilled

- exit code

137

- whether the crash happens before any jobs are submitted 5. Check node pressure too

kubectl describe node <node-name>

If the node is under memory pressure, that can contribute. Rule of thumb - Small cluster:

512Mi

may be enough - Medium/large cluster:

1Gi

or more may be needed for the worker If you want, I can help you narrow it down from your Helm values / worker manifest /

kubectl describe

output. Relevant docs: - Prefect workers - Kubernetes deployment examples - Work pools