https://prefect.io
Join Slack
Hey, is it possible to use context value as a default for Parameter?
# ask-community
d
Hey, is it possible to use context value as a default for Parameter?
m
I don't think so but perhaps you can set null as the default and then try to fetch the parameter value, if it is None then make use of prefect.context i.e.
Copy code
import random

from prefect import task, Parameter, Flow


@task(name="Task A")
def task_a(x):
    x = x or prefect.context.get("x")
    return x + 2


with Flow("test") as flow:
    x = Parameter("x", default=None)
    task_a(x)
z
Thanks for responding Marwan! I think that’s likely the best way to do this
👍 1
d
Thank you. This is great 🙂
👍 1
p
I am testing an approach subclassing the 
Parameter
class. I added another keyword argument that receives a callable that provides the default value if the parameter is null. I was using the approach above before but since it's so common in our flows, I wanted to eliminate the extra task. The advantage of not having a separate task is that the Parameter value can be used in templating without having to pass it as an explicit dependency to every task that needs it. You might argue that it's better to have an explicit data dependency, but in my use case the parameter is a date that is used as part of the path of the stored results.
m
Snap @Pedro Machado, I just implemented 
ContextDefaultParameter
and 
ContextDefaultDateTimeParameter
last night to solve this issue myself. I was planning on adding it to the Prefect Task Library at some point. I need to add the error handling code as I just wanted to see if I could get it to work.
p
Cool, Matt. How did you implement the interface?
m
Copy code
class ContextDefaultParameter(Parameter):

    def __init__(
        self,
        name: str,
        context_name: str,
        tags: Iterable[str] = None,
    ) -> None:
        super().__init__(name=name, required=False, tags=tags)
        self.context_name = context_name

    def run(self) -> Any:
        value = super().run()

        if value is None:
            value = prefect.context[self.context_name]

        return value
Note, I'm note a native python coder, there was much copy paste change going on here.
One thing I'd want to add to this is raising an error if the context does not have the named field.
p
Since you are accessing context with [] instead of 
get
, I should already raise an exception if the key is missing.
👍 1
m
Oh yeah, I forgot about the 
get
function. Even using 
get
though, i think it should raise an error if the context key could not be found.
p
Here's what I was playing with. I don't love the name :-)
Copy code
class ParameterWithDynamicDefault(Parameter):
    def __init__(
        self,
        name: str,
        required: bool = True,
        default_callable: Callable[[], JSONSerializableParameterValue] = None,
        tags: Iterable[str] = None,
    ) -> None:

        if default_callable and not callable(default_callable):
            raise ValueError(
                f"default_callable must be a callable. Got {type(default_callable)} instead."
            )

        self.default_callable = default_callable
        required = False if self.default_callable else required
        super().__init__(name, required=required, tags=tags)

    def run(self) -> Any:
        value = super().run()
        if value is None:
            value = self.default_callable()

        return value
m
Names are tough 😉
That's a different angle than I was thinking. The way that I read this functionality is like, "I'll use a value that the flow is called with, but if it's not supplied, I'll run this function to calculate a value".
p
Correct. I am using the scheduled start time from the context to compute a relative start date if one is not provided.
m
I'm doing a similar use case, use the scheduled_start_time if the date to use is not supplied.
This is how I use it 
extract_date = ContextDefaultDateTimeParameter(name="extract_date", context_name="scheduled_start_time")
p
Got it. I have to use some complex date offset logic and I wanted that encapsulated it in the 
Parameter
so I could use it in 
target
or 
location
templating.
m
Nice, I can see use cases for both approaches.
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by