haven
07/18/2021, 6:00 AMKubernetesSecret
should not rely on any upstream task/parameters. However, I think it might not be a great assumption. I'm using an env
parameter that denotes "DEV"
or "PROD"
and then would decide what secret I want to retrieve from k8s later (usually a database secret). Any plan to allow KubernetesSecret
to be a dynamic task and interpreted naturally in the flow without having to manually use flow.set_dependencies
?haven
07/18/2021, 6:06 AMdavzucky
07/18/2021, 8:31 AMhaven
07/18/2021, 8:35 AMenv = Parameter("env", default="DEV")
dt = Parameter("dt", default=date.today().strftime("%Y-%m-%d"))
use_proxy = Parameter("use_proxy", default=False)
env_config = config(env)
db = env_config["db"]
db_password = KubernetesSecret(db, "password", kubernetes_api_key_secret=None)
db_user = KubernetesSecret(db, "username", kubernetes_api_key_secret=None)
db_host = KubernetesSecret(db, "host", kubernetes_api_key_secret=None)
where
@task
def config(env: str) -> Dict[str, Any]:
...
davzucky
07/18/2021, 8:43 AMdavzucky
07/18/2021, 8:44 AMdavzucky
07/18/2021, 8:45 AMdavzucky
07/18/2021, 8:45 AMdavzucky
07/18/2021, 9:12 AMenv = Parameter("env", default="DEV")
dt = Parameter("dt", default=date.today().strftime("%Y-%m-%d"))
use_proxy = Parameter("use_proxy", default=False)
env_config = config(env)
db = env_config["db"]
K8sSecret = KubernetesSecret(kubernetes_api_key_secret=None)
db_password = K8sSecret(secret_name=db, secret_key="password")
db_user = K8sSecret(secret_name=db, secret_key="username")
db_host = K8sSecret(secret_name=db, secret_key="host")
davzucky
07/18/2021, 9:14 AMhaven
07/18/2021, 9:33 AMdavzucky
07/18/2021, 10:54 AM