hi :wave: When upgrading to `prefect 2.7.5` in our...
# prefect-community
hi 👋 When upgrading to
prefect 2.7.5
in our self hosted on kubernetes prefect I got the following error when starting a flow
Copy code
Submission failed. kubernetes.client.exceptions.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Audit-Id': 'b79d317b-f68f-46ff-b226-09adb2d37b66', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid': 'b89b22ad-0116-43a3-aa7b-0d04dc752da1', 'X-Kubernetes-Pf-Prioritylevel-Uid': '07475109-1722-457e-8c56-1612dc7046b5', 'Date': 'Wed, 04 Jan 2023 08:21:16 GMT', 'Content-Length': '349'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces \"kube-system\" is forbidden: User \"system:serviceaccount:prefect:prefect-conf-lfo\" cannot get resource \"namespaces\" in API group \"\" in the namespace \"kube-system\"","reason":"Forbidden","details":{"name":"kube-system","kind":"namespaces"},"code":403}
Nothing has changed in the cluster or service account and i I rollback to prefect
it works well Any idea / thoughts ?
Hey @Lucien Fregosi I'm not an expert on kubernetes but I believe this is actually related to some changes we made in a recent version https://github.com/PrefectHQ/prefect-helm/issues/91, relevant pr to address this https://github.com/PrefectHQ/prefect/pull/7864
@Mason Menges thanks for the input
Isn't that an important breaking change ?
It broke our Kubernetes Agent and I cannot find any relevant public documentation on what the new RBAC needs to be to work with the latest version. https://discourse.prefect.io/t/how-to-deploy-a-prefect-2-0-agent-to-a-local-kubernetes-cluster-and-connect-it-to-cloud-2-0-backend/979
Hey @Jean-Michel Provencher Thanks for bringing that up I raised it with the team so we can get the documentation updated https://github.com/PrefectHQ/prefect/issues/8120
Thank you, I'll rollback in the meantime I guess
I also ran into this isssue
👍 2