Thread
#prefect-community
    m

    Matt

    1 year ago
    Hi! I'm new to Prefect, working on my first flow, and struggling a bit with config and context in Prefect Core. I'm calling this from the command line and have the parameters going in fine and am grabbing secrets from env settings. I also understand that Prefect returns these as Tasks and not strings. Are both of these (Parameters and EnvVarSecrets) only available to the Flow? I don't seem to be able to access them from the Tasks themselves and it doesn't seem like they get automatically added to prefect.context or prefect.config. Is the only way to do this to pass them to the Task (socrata.download_dataset in this case) as parameters? I've tried adding the Secrets to both context and config programmatically but they don't seem to appear when I do this although the docs suggest this should work. My first attempt at a flow had parameters getting passed every which way and I'm trying to clean this up. Would love some guidance from the community. Thanks!
    nicholas

    nicholas

    1 year ago
    Hi @Matt - you can access any
    Secret
    task from within another task by calling its
    .get
    method, like such:
    token = Secret("SOME_SECRET").get()
    so a more full example might be:
    from prefect import Flow, task
    from prefect.client import Secret
    
    @task
    def my_task():
       my_secret = Secret("MY_SECRET").get()
       # do something with secret
    
    with Flow("My Flow") as flow:
      my_task()
    m

    Matt

    1 year ago
    hi nicholas, that looks like a good solution, but does it work if I'm running my flow from the command line? i'm doing a 'python myflow.py' approach. i'm not sure if the client only gets created if you do a prefect run flow or run it from a server. at least it just tried this approach and it couldn't find my secret.
    maybe i'm missing something else?
    Chris White

    Chris White

    1 year ago
    Hi Matt - did you set
    SOCRATA_TOKEN
    as an environment variable in the process that you ran
    python myflow.py
    from?
    @nicholas's suggestion uses a Prefect Secret specifically, which can be set via the
    PREFECT__CONTEXT__SECRETS__MY_SECRET
    environment variable, depending on which pattern you prefer
    m

    Matt

    1 year ago
    yep. calling
    token = EnvVarSecret("SOCRATA_TOKEN", raise_if_missing=True)
    from my 'with flow()' works fine. i can get the secret and pass it to my tasks as a parameter. i'm hoping there's a way to access it from my task either from context, config or something else so i don't have to explicitly pass it each time as a parameter. for instance i have a bunch of tasks that run against a database and they all need a DSN
    Chris White

    Chris White

    1 year ago
    Gotcha, in that case you can put it directly into context via environment variable:
    PREFECT__CONTEXT__SECRETS__SOCRATA_TOKEN=value
    and then pull it from
    prefect.context.secrets["SOCRATA_TOKEN"]
    within your tasks, without making it an explicit task itself
    m

    Matt

    1 year ago
    Perfect! This worked great. I'm just prefixing all my envs with PREFECT__CONTEXT__SECRETS__ and retrieving them in the tasks like you suggested. It seems there's not way to add secrets to context from code which probably makes sense from a security aspect as I think about it. My tasks param craziness is pretty much gone. Thanks!!