https://prefect.io logo
Join Slack
Powered by
If I pass a `Parameter` directly to a `DatabricksR...
# ask-community
n
If I pass a 
Parameter
directly to a 
DatabricksRunNow
task, Prefect detects the dependency and passes the information from the parameter to the task correctly. e.g.
Copy code
path = Parameter("path")

DatabricksRunNow(...)(
    databricks_conn_secret=SECRET,
    notebook_params: {
        "path": path,
    }.
)
However, if I plug the parameter into a formatted string, for example, the information is no longer passed from the parameter to the task correctly.:
Copy code
path = Parameter("path")

DatabricksRunNow(...)(
    databricks_conn_secret=SECRET,
    notebook_params: {
        "path": f"{path}",
    }.
)
In this case, the parameter class instance is plugged into the string vs. the actual parameter value that we want. So the notebook is given a 
repr()
of a 
Parameter
class instance — which is unusable, of course — instead of the string value of the parameter that we actually want. Why is that, and is there a way around this?
My use case: I am trying to build a string out of flow parameter values and pass that string as an input notebook parameter. I could change the notebook to accept the raw parameter values and build the string inside the notebook, so I can do that if necessary. It just seemed more natural in my case to build the string as part of the flow pass the already-built string to the notebook.
k
Hey @Nicholas Chammas, I think you need an intermediate task to do this because that f-string is evaluated during build time and you need something evaluated at runtime.
👍 1
Maybe you can do something like 
"path": task(lambda x: f"{x}")(path)
n
Hmm, OK. Just so I understand: How come that’s not necessary in the first example? Is this where Prefect is doing the extra work by generating those intermediate 
List
and 
Dict
tasks?
k
The Parameter is a task also so it’s also evaluated at runtime. Stuff like list access or dictionary access 
(x[0], x["test"])
are also evaluated immediately like the 
f-string
so the execution of these is not deferred so you can’t do these in the Flow block if 
x
is the output of a task inside the Flow.
2 Views
Open in Slack
PreviousNext
Powered by