https://prefect.io logo
Title
p

Pedro Machado

09/21/2020, 12:45 AM
Hi there! I have a flow with Parameters
start_date
and
end_date
. If these are not provided, I want to use the
scheduled_start_time
from the context to generate the
start_date
and
end_date
. What is the cleanest way to write this logic given the fact that the parameters will only be available at run time? I currently have a task that takes both parameters and outputs a tuple of dates. This tuple has either the value of the parameters converted to a
pendulum
date or the values generated from the
prefect.context.scheduled_start_time
also as dates. Is there a better way? This is what the flow looks like now.
k

Kyle Moon-Wright

09/21/2020, 5:40 PM
Hey @Pedro Machado, Digging into this a bit, I think the best way to pass context objects to parameter defaults is to set up a task that grabs those values and use them as an upstream task like so:
from prefect import Flow, task, Parameter

@task
def get_date():
    import prefect
    return prefect.context.scheduled_start_time

@task
def do_stuff(data):
    print(f"Doing stuff with the Parameter task: {data}")

with Flow("My Flow") as flow:
    time = get_date()
    start_time = Parameter(name="Start Time", default=str(time))
    do_stuff(start_time)

flow.run()
Hopefully that makes sense and at least takes a step towards what you're looking for. I'm not sure if this setup would be conducive/useful for your flow.
p

Pedro Machado

09/22/2020, 3:20 AM
Thanks, Kyle. I was able to run your example, but could not extend it to my use case. I was getting some serializations errors. Also, when you pass the task as an argument to
default
it can't figure out the dependencies. I tried to add a dependency via
set_upstream
and got
ValueError: Parameters must be root tasks and can not have upstream dependencies.
I reverted back to my original approach but the thing I don't like about it is that I cannot access the modified parameters through the context. I'd like to use the start/end date to template a task's result location. I'll post another question specifically about accessing upstream inputs when templating the result location.