Thanks for the details — this is a namespace/RBAC mismatch. What’s happening - Your Kubernetes worker is running as the service account

system:serviceaccount:prefect:prefect-worker

(in the

prefect

namespace). - The worker is trying to create Jobs in the

default

namespace. By default, that SA only has permissions in its own namespace (

prefect

), so the API denies the request. Where “default” is coming from - In Prefect 3.x with the Kubernetes worker, the Job namespace defaults to

default

unless you set it in one of these places (highest precedence first): 1) deployment

job_variables["namespace"]

2) your Kubernetes work pool’s base job template (the “namespace” variable) 3) default (“default”) Note: The Helm value

worker.config.jobNamespace

does NOT automatically override the work pool/deployment job namespace. It primarily influences what the worker observes and defaults, but the submitted Job’s namespace is read from the work pool/deployment template. Two solid ways to fix it Option A — Run jobs in the

prefect

namespace (recommended): - Set the namespace on your deployment (quickest):

deployment = RunnerDeployment(
    name="fraio-trigger",
    flow_name="trigger-flow",
    entrypoint="fraio_orchestrator.fraio_trigger_flow.trigger_flow",
    work_pool_name="fraio-pool",
    job_variables={
        "image": "harbor.mgmt.vintecc.cloud/vintecc-ags-hbm/fraio_algo:local",
        "namespace": "prefect",
        "service_account_name": "prefect-worker",
        "env": {
            "PREFECT_API_URL": "<http://prefect-server.prefect:4200/api>",
            "PREFECT_DB_PASSWORD": os.environ["PREFECT_DB_PASSWORD"],
            "PREFECT_DB_USER": os.environ["PREFECT_DB_USER"],
            "PREFECT_DB_NAME": os.environ["PREFECT_DB_NAME"],
            "PREFECT_DB_HOST": os.environ["PREFECT_DB_HOST"],
            "PREFECT_DB_PORT": os.environ["PREFECT_DB_PORT"],
        },
    },
    version="local",
    tags=[],
    description=None,
)

- Or set it once at the work pool level so all deployments inherit it: - In the UI: Work Pools → open

fraio-pool

(must be type “kubernetes”) → Configure → set Namespace =

prefect

. - Or via CLI with a base job template (you can update an existing pool): 1) Inspect current pool:

prefect work-pool inspect fraio-pool

2) Create a small JSON file (e.g.,

k8s-template.json

) with a default namespace:

{
  "variables": {
    "namespace": {
      "default": "prefect"
    }
  }
}

3) Apply it (create can also update when used with overwrite):

prefect work-pool create "fraio-pool" --type kubernetes --base-job-template ./k8s-template.json --overwrite

Option B — Keep jobs in

default

and grant RBAC there: - Create a Role/RoleBinding in the

default

namespace so

prefect:prefect-worker

can create Jobs:

apiVersion: <http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>
kind: Role
metadata:
  name: prefect-worker
  namespace: default
rules:
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["get","list","watch","create","update","patch","delete"]
- apiGroups: [""]
  resources: ["pods","pods/log","events"]
  verbs: ["get","list","watch"]
---
apiVersion: <http://rbac.authorization.k8s.io/v1|rbac.authorization.k8s.io/v1>
kind: RoleBinding
metadata:
  name: prefect-worker
  namespace: default
roleRef:
  apiGroup: <http://rbac.authorization.k8s.io|rbac.authorization.k8s.io>
  kind: Role
  name: prefect-worker
subjects:
- kind: ServiceAccount
  name: prefect-worker
  namespace: prefect

Sanity checks - Confirm your work pool type and current template:

prefect work-pool inspect fraio-pool

- Verify RBAC from the worker SA:

kubectl auth can-i create jobs --as=system:serviceaccount:prefect:prefect-worker -n default
kubectl auth can-i create jobs --as=system:serviceaccount:prefect:prefect-worker -n prefect

A couple of small notes - Ensure

fraio-pool

is a Kubernetes work pool. If it’s not, create it:

prefect work-pool create "fraio-pool" --type kubernetes

- In most cases, your flows only need `PREFECT_API_URL`; they don’t need DB credentials. Not harmful, just not required for flow runs. If you can share: - The output of

prefect work-pool inspect fraio-pool

(especially the “namespace” variable) - Whether you’d prefer to run jobs in

prefect

or

default

…I can tailor the exact steps/yaml for your setup.