https://prefect.io logo
Title
m

Matthias

06/01/2020, 8:55 PM
Dear Prefect Community šŸ‘‹ I’d like to make a Kubernetes secret (containing GCP credentials) available in the worker pods created by the Kubernetes agent. Is this possible?
šŸ‘‹ 2
k

Kyle Moon-Wright

06/01/2020, 9:18 PM
Hey @Matthias! Let me ask the team about best practices here.
Gained some insight: A potential solution could be to utilize a KubernetesJobEnvironment and define the job spec that uses the secret from the get-go. More information about implementing this environment and what the yaml may look like can be found in the documentation.
m

Marwan Sarieddine

06/02/2020, 12:14 AM
@Matthias - if you are using Dask you need to create the k8s secret in the same namespace as the worker, and pass the credentials as environment variables in the worker_spec.yaml - something like this under spec.containers[]
env:
      - name: SECRET_USERNAME
        valueFrom:
          secretKeyRef:
            name: mysecret
            key: username
      - name: SECRET_PASSWORD
        valueFrom:
          secretKeyRef:
            name: mysecret
(I had done something similar to authenticate with AWS - I presume GCP should be similar)
m

Matthias

06/02/2020, 6:30 AM
@Marwan Sarieddine, @Kyle Moon-Wright thank you so much for your help! The KubernetesJobEnvironment is indeed what I was looking for. Seems nothing is impossible with Prefect šŸ™‚
:marvin: 1
šŸš€ 1