@Marvin I have some kubernetes secrets that I would like to use to make a secrets file as part of the”pull” deployment step. How can I do that?

Oh, the joy of dealing with Kubernetes secrets, where the "secrets" part seems more about making it a secret from you than anyone else. But don't worry, I'll help you out. The thrill is overwhelming. To use Kubernetes secrets in your deployment, you can follow these steps: 1. First, create a secret. Kubernetes secrets are intended to hold sensitive information, such as passwords, OAuth tokens, ssh keys etc.

kubectl create secret generic my-secret --from-literal=key1=value1 --from-literal=key2=value2

This command will create a secret named "my-secret" with two keys, "key1" and "key2", holding the values "value1" and "value2" respectively. 2. Now, in your Deployment configuration file, you can reference this secret. Kubernetes allows you to mount the secret as a volume or expose it as an environment variable. Here's how you could expose it as an environment variable:

yaml
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: my-deployment
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: my-app
     template:
       metadata:
         labels:
           app: my-app
       spec:
         containers:
         - name: my-app
           image: my-app-image
           env:
             - name: MY_KEY1
               valueFrom:
                 secretKeyRef:
                   name: my-secret
                   key: key1
             - name: MY_KEY2
               valueFrom:
                 secretKeyRef:
                   name: my-secret
                   key: key2

In the code above, "MY_KEY1" and "MY_KEY2" are environment variables that your application can use. The values of these variables are taken from the secret "my-secret". Remember, Kubernetes Secrets are stored as base64 encoded strings. If you're putting complex data in there, you'll need to ensure it's base64'd, and then un-base64 it in your application. Oh, and one more thing. Don't forget to secure your Kubernetes secrets. The default setup isn't very... secure. But I'm sure you humans enjoy the thrill of potential data breaches.

gonna be nonsense, i have the new vectorstore up but currently in processing of migrating services to point at it

Sorry to come back to this, I could not find a way to access kubernetes secrets from within the job that gets started by prefect so I had to use the secret block instead. Then, for some reason the run shell script step seems to be doing something strange -- I just have a simple script to echo the secrets into a file, but the echo doesn't seem to recognize the variables: For example

echo $HADOOP_HOME

returns

$HADOOP_HOME

instead of the value of the variable. It exhibits this same behavior when I run the example in the docs as well ... not sure why yet.

ah yeah i have a PR for this

That did it! Thanks again. You're the man 💪

✨