Tomas Moreno
06/17/2024, 6:16 PMMarvin
06/17/2024, 6:16 PMprefect.yaml
. This way, you can inject secrets without exposing them in your configuration.
Below are some useful links from the Prefect documentation related to managing job variables:
- Overriding job variables
- Prefect deployment guide
- GCP Worker Guide
Using these resources, you should be able to find a method to keep secrets safe while being used in Prefect deployments.Nate
06/17/2024, 6:17 PMI have an env var I'm setting from a prefect secret in my prefect.yamlhow are you referencing that in the prefect yaml? if you reference the secret directly, it shouldnt show up in the UI and the actual value will be rendered when needed
Tomas Moreno
06/17/2024, 6:22 PMdefinitions:
work_pools:
work_pool_stg: &work_pool_stg
name: my-ecs-pool
work_queue_name:
job_variables: &work_pool_stg_job_variables
env: &work_pool_stg_env_variables
REDSHIFT_PW: "{{ prefect.blocks.secret.dbt-non-prd }}"
Nate
06/17/2024, 6:22 PMTomas Moreno
06/17/2024, 6:23 PMNate
06/17/2024, 6:24 PMTomas Moreno
06/17/2024, 6:25 PMNate
06/17/2024, 6:26 PMTomas Moreno
06/17/2024, 6:33 PMNate
06/17/2024, 6:33 PM