Giuliano Mega

01/19/2023, 1:09 PM
Hello there folks! I'm a prefect cloud user running agents on GKE autopilot. I've recently updated my prefect agent from 2.7.0 to 2.7.8 to be able to deal with (was getting thousands of those every day), and am now struggling with an issue with the new agent, namely (permission denied while trying to read the
namespace). As per the workaround in the bug, I've updated my existing staging agent role so that it can read namespaces:
āžœ ~ kubectl describe role prefect-staging-agent 
Name:         prefect-staging-agent
Labels:       <none>
Annotations:  <none>
  Resources    Non-Resource URLs  Resource Names  Verbs
  ---------    -----------------  --------------  -----
  jobs.batch   []                 []              [list watch create update patch get delete]
  namespaces   []                 []              [list watch get]
  pods/log     []                 []              [list watch get]
  pods/status  []                 []              [list watch get]
  pods         []                 []              [list watch get]
to no avail. Still getting
"Failure","message":"namespaces \"kube-system\" is forbidden: User \"system:serviceaccount:default:default\" cannot get resource \"namespaces\" in API group \"\" in the namespace \"kube-system\"","reason":"Forbidden"
Any ideas of what might I be missing? Guess my main concern is that kube-system might be off limits for autopilot but couldn't find anything saying you can't read stuff from it.
āœ… 1
OK, given that I'm deploying agents with Terraform, I'm trying a workaround here where I read the uid from the kube-system namespace and then pass that to the agent via PREFECT_KUBERNETES_CLUSTER_UID, which I just discovered exists. šŸ™‚ Let's see if that works.
success šŸ˜…
šŸ™Œ 1
for reference:
data "kubernetes_namespace" "kube_system" {
  metadata {
    name = "kube-system"

# To access the UID:

:thank-you: 1