If I wish to inject values into my flow at run time from the environment (env vars) where should this occur? Should it be at the start of my flow using a Context context manager? These are values such as the application env (prod, dev, staging) and particular business logic values that are not secrets. I tried to create a user defined config object and then have my tasks read from them but then realised that this object was being created at flow build time and so the values were being frozen to what I had either locally when I registered my flow or in my build environment.
Hi @Hawkar Mahmod! RunConfig and setting environment variables there may help you. I have also seen people do this using default Parameters set on projects on Prefect Cloud.
h
Hawkar Mahmod
04/19/2021, 7:23 AM
Apologies I think I mis-framed the question. I am doing as @Kevin Kho and @Matthew Blau say but I’m not sure what is the best way to make these available during my flow runs. Should I utilise the env vars directly in my tasks using os.environ or load the values up in the Prefect context. If the latter where would you set these values on the context? In a separate task or in the Flow context itself?
k
Kevin Kho
04/19/2021, 1:39 PM
If it’s a parameter, you can grab it inside the flow like this. You can use os.environ or the config .
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.