Need an advice on how to use context in cloud Hey there! Glad to be here :-) We use Prefect via Cl...

Sasha Mikhailov

01/14/2021, 10:12 AM

Need an advice on how to use context in cloud Hey there! Glad to be here :-) We use Prefect via Cloud. There some parameters in our flows. I want to store parameters' values in the prefect.context. Code from the docs works fine locally:

Copy code

@task(log_stdout=True)
def print_context():
  print('print context from task:', prefect.context.get('db_env'))   
   
DB_ENV = Parameter("database to connect to", default='local')

prefect.context['db_env'] = DB_ENV  # ← doesn't work in cloud

with Flow('testing_context') as flow:
  print_context(upstream_tasks = [DB_ENV])
      
flow.run()

then I register the flow in a docker storage and push it to the cloud. And it doesn't work when I run the flow via cloud ui: it shows None instead of parameter.

Sasha Mikhailov

01/14/2021, 11:20 AM

In general, is it possible (or good practice) to "pack" several parameters to a context and then call it inside tasks? In case of using in Cloud (locally it works fine).

Spencer

01/14/2021, 1:31 PM

Accessing the context outside of a task won't persist to a non-local flow since the context isn't deployed with the pickled flow. I've used Secrets for this to store credentials for databases and other services https://docs.prefect.io/orchestration/concepts/secrets.html

👀 1

Zanie

01/14/2021, 4:20 PM

If I’m not mistaken, all of the parameters in your flows should be able in the context at

prefect.context.parameters

Zanie

01/14/2021, 4:23 PM

For example

Copy code

import prefect
from prefect import task, Flow, Parameter


@task(log_stdout=True)
def display():
    print(prefect.context.get("parameters"))


with Flow("with-parameters") as flow:
    foo = Parameter("foo", default=1)
    bar = Parameter("bar", default=2)
    flow.add_task(foo)
    flow.add_task(bar)
    display()

flow.run()

🎯 1

👀 1

🙌 1

Sasha Mikhailov

01/15/2021, 7:16 AM

@Zanie wow, Michael, thanks a lot! That's the thing! I saw that method in docs, but I tried to apply it in wrong way >_< Need to pay more attention :-)

Zanie

01/15/2021, 4:19 PM

Glad it worked 🙂

Open in Slack

Previous Next

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.