https://prefect.io logo
s

Stefan

08/16/2023, 8:01 AM
Hi! Where do you store secrets such as database-connections trings that's usually stored in an .env-file? I don't want to add them in cleartext when setting up a pool or a Process.
r

redsquare

08/16/2023, 8:05 AM
prefect has secret blocks else depends on the infra you are running on, k8s/aws secret manager etc etc
m

morten

08/16/2023, 8:17 AM
You can take a look at this: https://docs.prefect.io/2.11.3/concepts/blocks/ it might help
s

Stefan

08/16/2023, 8:30 AM
Thanks! But I get this error?
m

morten

08/16/2023, 8:43 AM
not sure what to do, have not used secrets in the code, only as a part of the prefect.yaml file
s

Stefan

08/16/2023, 8:43 AM
OK thanks anyway. How do you do it in the yaml-file? I'm assuming you don't store passwords there?
m

morten

08/16/2023, 8:44 AM
it depends on the secret, I suggest reading the deployment section of the docs
s

Stefan

08/16/2023, 8:56 AM
It's not really making it any clearer 😞 I see an example of pulling a secret when git cloning (which've used), but for setting variables for the flow itself - I don't see anything.
r

redsquare

08/16/2023, 8:59 AM
did you create a secret block called t when you ran the code above?
s

Stefan

08/16/2023, 9:00 AM
Yeah, I got that working despite the error in PyCharm.
So I can do it like that, but I'm wondering if I can also specify the env-variables in the prefect.yaml deployment file and link to the secret there, so that it's available at the flows runtime.
I may have found it.
r

redsquare

08/16/2023, 9:06 AM
cool, we use k8s secrets that are injected using env vars
upvote 1
s

Stefan

08/16/2023, 9:07 AM
Mind showing me how it looks like?
r

redsquare

08/16/2023, 9:24 AM
in terms of the job template it does this, which asically pulls in all the secrets from the prefect-flow-secrets group in k8s
Copy code
customizations:
  - op: add
    path: /spec/template/spec/containers/0/envFrom/-
    value:
      secretRef:
        name: prefect-flow-secrets
so it turns all of the secrets in that group into environment variables for the container
c

Christopher Boyd

08/16/2023, 6:03 PM
I’ve done this as well with managed identity for both ACI and AKS (retrieving from azure keyvault), but presumably doable with secrets manager for GCP and AWS as well