https://prefect.io logo
Title
n

Nic

12/07/2022, 11:03 AM
Are there any best practices when it comes to setting up a dev and production env in prefect? I was thinking to put it in two different workspaces, so it would also be easier to take advantage of blocks, but I'm not sure if i could run two agents on two different workspaces on the same pc. (Would have to login to two different workspaces simultanously?) Currently i've setup two different docker blocks that injects the secrets i need in different environment into the env vars in the container, but this would lead to exposed secrets if i was to inject secrets into the docker like this. One way would be to use different Azure vaults and just inject these into the docker, but I believe the prettier solution would be to use prefect blocks. Anyone who did something similar and can share ideas or experiences? TL:DR - What is the best way to deal with secerts/connection string and other variables that span across environments? Ex.. The Server1 Var in my Dev environment == DevServer Server1 Var in Prod var == ProdServer
r

redsquare

12/07/2022, 1:07 PM
we use separate environments in github actions, the deployments auto patch in the correct vault environment secrets
n

Nic

12/07/2022, 1:08 PM
What about secrets used in Python flow itself?
r

redsquare

12/07/2022, 1:09 PM
they get patched in to the k8's job template
n

Nic

12/07/2022, 1:12 PM
Where from? You deploy them with them the vars? Current we have different values for same var locally,dev and prod and want the tasks to be able to grab them.
So your GitHub actions inject the var into the flow as you deploy the code?
r

redsquare

12/07/2022, 1:23 PM
no the github action runs the deployment.py that deploys the flow and creates the customizations needed to the job template - which when runs pulls in the secrets
for secret in secret_stores:
secretPatch = {
"op": "add",
"path": "/spec/template/spec/containers/0/envFrom/-",
"value": {
"secretRef": {
"name": secret.lower()
}
}}
customizations.append(secretPatch)
so that will pull in the secrets from that 'vault' in our case rancher secrets (k8s)
and we have 3 separate secret stores - global, project level and flow level