Hi! I'm new to Prefect, working on my first flow, ...
# prefect-community
Hi! I'm new to Prefect, working on my first flow, and struggling a bit with config and context in Prefect Core. I'm calling this from the command line and have the parameters going in fine and am grabbing secrets from env settings. I also understand that Prefect returns these as Tasks and not strings. Are both of these (Parameters and EnvVarSecrets) only available to the Flow? I don't seem to be able to access them from the Tasks themselves and it doesn't seem like they get automatically added to prefect.context or prefect.config. Is the only way to do this to pass them to the Task (socrata.download_dataset in this case) as parameters? I've tried adding the Secrets to both context and config programmatically but they don't seem to appear when I do this although the docs suggest this should work. My first attempt at a flow had parameters getting passed every which way and I'm trying to clean this up. Would love some guidance from the community. Thanks!
Hi @Matt - you can access any
task from within another task by calling its
method, like such:
Copy code
token = Secret("SOME_SECRET").get()
so a more full example might be:
Copy code
from prefect import Flow, task
from prefect.client import Secret

def my_task():
   my_secret = Secret("MY_SECRET").get()
   # do something with secret

with Flow("My Flow") as flow:
hi nicholas, that looks like a good solution, but does it work if I'm running my flow from the command line? i'm doing a 'python myflow.py' approach. i'm not sure if the client only gets created if you do a prefect run flow or run it from a server. at least it just tried this approach and it couldn't find my secret.
maybe i'm missing something else?
Hi Matt - did you set
as an environment variable in the process that you ran
python myflow.py
@nicholas's suggestion uses a Prefect Secret specifically, which can be set via the
Copy code
environment variable, depending on which pattern you prefer
yep. calling
Copy code
token = EnvVarSecret("SOCRATA_TOKEN", raise_if_missing=True)
from my 'with flow()' works fine. i can get the secret and pass it to my tasks as a parameter. i'm hoping there's a way to access it from my task either from context, config or something else so i don't have to explicitly pass it each time as a parameter. for instance i have a bunch of tasks that run against a database and they all need a DSN
Gotcha, in that case you can put it directly into context via environment variable:
Copy code
and then pull it from
within your tasks, without making it an explicit task itself
🙌 1
Perfect! This worked great. I'm just prefixing all my envs with PREFECT__CONTEXT__SECRETS__ and retrieving them in the tasks like you suggested. It seems there's not way to add secrets to context from code which probably makes sense from a security aspect as I think about it. My tasks param craziness is pretty much gone. Thanks!!
🎉 1