b

    Brad I

    1 year ago
    We’re using the k8s agent with dask executor on GKE with workload identities. Does anyone know if it is still required to set the
    GCP_CREDENTIALS
    secret in Prefect Cloud? Or will it just use the service account tied to the namespace? The latter would be nice to avoid creating any long lived service account keys.
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Brad I the GCP stuff in Prefect look for the
    GCP_CREDENTIALS
    under secrets and use that to authenticate so I don’t think it will use the service account tied to the namespace automatically.
    Someone on the team will doublecheck this
    b

    Brad I

    1 year ago
    Thanks! Was wondering if it would fall back to use the injected gcloud auth config. I know this is probably very specific to GKE and not general prefect.
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey I dug through the code more. I think this will work. See this
    b

    Brad I

    1 year ago
    @Kevin Kho we’ve been testing this today and it does seem to work with the secret deleted from Prefect Cloud. Our flow/task code is able to use the workload identity permissions to access bigquery, GCS, and firestore correctly since the underlying google libraries support it. This is great because it’s one less manual set up step (we use pulumi for infrastructure automation), easier to set up multiple k8s namespaces, we don’t have static keys floating around, and takes advantage of short-lived tokens.https://cloud.google.com/blog/products/containers-kubernetes/introducing-workload-identity-better-authentication-for-your-gke-applications
    lol, yeah, good timing
    Kevin Kho

    Kevin Kho

    1 year ago
    Welp looks like we got back at the same time 😅. Glad you got it working.