Quick question on best practices: I'm trying to use "Secrets" with a ShellTask and see two possibilities: 1. Get the Secret when I register the flow, eg:

from prefect import Flow
from prefect.tasks.shell import ShellTask
from prefect.client import Secret

environment = {}
secret_key = Secret("SECRET_KEY")
environment['SECRET_KEY'] = secret_key.get()

with Flow(name, schedule=schedule) as flow:
  task(command='./do-the-thing', env=environment)
  flow.register(project_name=project_name)

2. Use

prefect.client.Secret

to get the Secret from within the Task when it starts (similar to how an entrypoint script is often used to set environment variables in Docker environments). eg:

# register-tasks.py
from prefect import Flow
from prefect.tasks.shell import ShellTask

with Flow(name, schedule=schedule) as flow:
  task(command='./do-the-thing')
  flow.register(project_name=project_name)

# do-the-thing
#!/usr/bin/env python3
from prefect.client import Secret

secret_key = Secret("SECRET_KEY")
os.environ['SECRET_KEY'] = secret_key.get()

Is one of these a recommended over the other as the "best practice" for Prefect?

Hi Jason - yes, pulling the secret at runtime (option 2) is preferred over pulling it at build time (option 1). With option 1 you run the risk of storing the value of the secret along with the Flow, which is generally not a good idea, and ultimately defeats the purpose of using a Prefect Secret at all. Relatedly, it seems you are using

os.environ

to update your local environment; note that the environment that your flow runs in could be different from the environment that you built your flow in. Moreover, setting OS env vars directly in python is generally not good practice (the environment variables only take affect for subprocesses spawned from the parent process where

os.environ

was updated)

How do you recommend setting the environment variables at run time? Or do you recommend not using environment variables for the value of the secret?

@Jason Nochlin under the hood, a Secret is a prefect task whose job it is to retrieve the secret