https://prefect.io logo
j

Jasono

11/24/2020, 6:00 PM
Hi does the Context {…} defined in the Prefect Cloud web UI (I’m referring to the Run screen where Parameters and Context are manually entered) override the same context variables in config.toml? It appears to me the context entered in the web UI is ignored.
j

josh

11/24/2020, 6:52 PM
Hi @Jasono certain values present in context at run time will be overwritten after the user provided context is loaded (i.e. flow id, flow run id, scheduled start time) because these values are used for the runner during execution of the flow. Relevant code where context is loaded: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/engine/cloud/flow_runner.py#L372-L380
j

Jasono

11/24/2020, 6:55 PM
Sorry I wasn’t clear. These are user defined variables that are overriden, not those you mentioned.
j

josh

11/24/2020, 6:57 PM
Not sure if I follow, if I set context variables in the web UI I see them being populated in the flow run. 🤔 What are you attempting?
j

Jasono

11/24/2020, 6:58 PM
I want to pass “reporting period” for instance
don’t want to use Parameter for that.
Do you also define those user defined context variables in your config.toml?
I do, because otherwise the flow script fails when I try to register it
j

josh

11/24/2020, 7:02 PM
How are you accessing the variable from context? Registration does not run the flow. If you’re doing something like this:
Copy code
with Flow():
   my_task(context.get("reports"))
That will not work because context is being evaluated at initialization of the flow object before registration. Subsequently the context provided by the UI will not be read in there because that code is running prior to the flow run. Instead your context should be accessed inside of a task
Copy code
@task
def my_task():
   prefect.context("reports")
j

Jasono

11/24/2020, 7:06 PM
I’m trying to do it like your first example, and it is necessary as I dynamically create tasks within the Flow block.
Copy code
with Flow("bdx_process_9xx") as flow:

    acctDate = str(prefect.context.acctDate)

    prodDate = str(prefect.context.prodDate)

    runDesc9xx = prefect.context.runDesc9xx

    print(f'ProdDate={prodDate} ..{runDesc9xx}')

    

    for queryKey, (queryCmd, dependQry) in cmdPairs(acctDate, prodDate, runDesc9xx).items():                      

        task = BDX_Task(name=queryKey)(queryKey =queryKey,

                        queryCmd=queryCmd

                        )

        flow.add_task(task)

 

        if len(dependQry) > 0 :

            if type(dependQry) is list:

                for q in dependQry:

                    flow.get_tasks(name=queryKey)[0].set_upstream(flow.get_tasks(name=q)[0])

            else:

                flow.get_tasks(name=queryKey)[0].set_upstream(flow.get_tasks(name=dependQry)[0])
j

josh

11/24/2020, 7:15 PM
Yeah in your snippet you are grabbing values from your local context and passing them into the tasks you create therefore they are static and cannot be changed. If you want to access a dynamic context value at runtime instead of build time then you will need to grab it from context in a similar way to the snippet I provided above.
j

Jasono

11/24/2020, 7:16 PM
okay. thank you!
3 Views