Hey, how can I provide secrets to the Kubernetes A...
# ask-community
j
Hey, how can I provide secrets to the Kubernetes Agent? I noticed that I can set some envs at
KubernetesRun
but these are only strings. Is there currently an easy way to provide secrets as envs? My intuition says I have to provide a custom job yaml then, is that correct, or am I missing an easier way?
1
j
If you want to provide secrets for your flow, you'd need to provide a custom job yaml. This could be either set as part of
KubernetesRun
, or on the agent as the default job yaml (with
--job-template
).
The default template is here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/agent/kubernetes/job_template.yaml (basically an empty job template with a single container), you can add whatever secrets you need.
s
I've been using [PodPreset](https://kubernetes.io/docs/tasks/inject-data-application/podpreset/#use-pod-presets-to-inject-environment-variables-and-volumes) to pass a slew of common environment vars to each Pod that's spawned. I wouldn't use this for sensitive information, but it allows you to skip the two-step process of using the
--job-template
just to define them.
j
but it allows you to skip the two-step process of using the
--job-template
just to define them.
Oh neat. Note that for simple environment variables (not secrets) you can use the
--env
cli flag instead if you're setting them at the agent level - no need to add a custom job template just for environment variables.
j
Kay thanks 🙂 So the job template you provided is filled in at runtime, with the correct image and stuff?
j
yeah. Prefect will fill in the first container in the job pod spec with all the prefect required stuff.
So you only need to add whatever extra things you want.
p
@Jim Crist-Harif why there is an argument option on KubernetesRun to provide the image_pull_secret if it is not present in the default job template?
n
Before I'd add
image_pull_secret
in
DaskKubernetesEnvironment
and it would work. also I'd add
image_pull_secret
in kubernetes agent. Now like @Pedro Martins I add it in KubernetesRun but it doesn't work. So now the only way is through job_template?
j
It doesn't need to be present in the job template, it will automatically be added if set on either the
KubernetesRun
object or on the agent (note that the
Secret
itself needs to already exist in the namespace). You do need to ensure that the version of the k8s agent you're running is up to date, or that feature will be ignored. If this is not working for you, this is a separate issue than Joel's above - please open a new thread.
j
Thanks @Jim Crist-Harif, seems like nice flexible solution then 🙂