https://prefect.io logo
s

Samay Kapadia

01/19/2022, 8:59 AM
Hi prefects. I’m trying to execute a RunNamespacedJob task in my kubernetes setup but I’m running into this error
Copy code
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "jobs.batch \"dummy\" is forbidden: User \"system:serviceaccount:default:default\" cannot get resource \"jobs/status\" in API group \"batch\" in the namespace \"default\"",
  "reason": "Forbidden",
  "details": {
    "name": "dummy",
    "group": "batch",
    "kind": "jobs"
  },
  "code": 403
}
For context, I’ve applied the yaml from
prefect agent kubernetes install --rbac
so all the permissions should work in theory. I'm stuck at what could be wrong
also attached relevant role and role binding
Quick update, this works if I change the role rules to contain
*
instead of
jobs
, so there is something fishy goign on
a

Anna Geller

01/19/2022, 11:47 AM
Nice work figuring this out! Regarding
*
, I think this makes sense because the task does multiple things: • it creates namespaced job • it then reads job status to check on it • finally it lists pods to get the exact pod name to read its logs and to delete it in the end So the job status permission was missing, which explains why it works when you use wildcard instead.
s

Samay Kapadia

01/19/2022, 11:47 AM
Ah okay
I would really like to not use
*
, so adding job status should work?
And in that case
prefect agent kubernetes install --rbac
should be amended to fix this
a

Anna Geller

01/19/2022, 11:48 AM
Yeah, you can check the code I shared and see there all the API action this task does. Reading logs + deleting a job may need another entries
s

Samay Kapadia

01/19/2022, 11:49 AM
Thanks Anna! Let me see if this works
👍 1
a

Anna Geller

01/19/2022, 11:49 AM
The default RBAC only provides permissions to spin up flow runs as kubernetes jobs, it doesn’t provide permissions for other Kubernetes tasks
s

Samay Kapadia

01/19/2022, 11:57 AM
Okay it worked when I added
jobs/status
as a resource in the role definition
Thanks again for the help catjam
The rules in yaml for anyone interested
Copy code
rules:
  - verbs:
      - '*'
    apiGroups:
      - extensions
      - batch
    resources:
      - jobs/status
      - jobs
  - verbs:
      - '*'
    apiGroups:
      - ''
    resources:
      - events
      - pods
a

Anna Geller

01/19/2022, 12:48 PM
Thank you so much for sharing! 🙌
3 Views