https://prefect.io logo
m

Miecio

08/28/2020, 10:34 AM
Hello I have some problem with Prefect/Dask and secrets, I want to use PostgreSQL tasks from task library, so I need to pass somehow Secret task containing pg password. I thought that using EnvVarSecret will be enough, but seems that env variables are not propagated from prefect agent to dask workers, so my password is always empty. Do you know what solution can be used? or maybe can I assingn those EnvVarSecret tasks to run inside my agent job (where I have all required envs)?
👀 1
j

Jim Crist-Harif

08/28/2020, 1:43 PM
Hi @Miecio, you need to ensure the environment variable was set in the dask workers as well (e.g. if you're on k8s, you'd need to set it in the worker pods), or you'd need to use a
PrefectSecret
instead. Alternatively, if you're using prefect cloud you can store secrets in prefect cloud directly, and your workers can securely access them without needing to modify the worker environment. See https://docs.prefect.io/core/concepts/secrets.html#prefect-cloud for more info.
m

Miecio

08/28/2020, 1:44 PM
can I use
PrefectSecret
without prefect cloud? as I'm required to have all stack on prem
j

Jim Crist-Harif

08/28/2020, 1:46 PM
You can, but in that case it will load secrets from the local prefect config (putting you in the same situation as when reading secrets from environment variables).
m

Miecio

08/28/2020, 1:51 PM
ok, thanks for Your response 🙂