Pedro Machado
09/21/2020, 12:45 AMstart_date
and end_date
.
If these are not provided, I want to use the scheduled_start_time
from the context to generate the start_date
and end_date
.
What is the cleanest way to write this logic given the fact that the parameters will only be available at run time?
I currently have a task that takes both parameters and outputs a tuple of dates.
This tuple has either the value of the parameters converted to a pendulum
date or the values generated from the prefect.context.scheduled_start_time
also as dates.
Is there a better way?
This is what the flow looks like now.Kyle Moon-Wright
09/21/2020, 5:40 PMfrom prefect import Flow, task, Parameter
@task
def get_date():
import prefect
return prefect.context.scheduled_start_time
@task
def do_stuff(data):
print(f"Doing stuff with the Parameter task: {data}")
with Flow("My Flow") as flow:
time = get_date()
start_time = Parameter(name="Start Time", default=str(time))
do_stuff(start_time)
flow.run()
Hopefully that makes sense and at least takes a step towards what you're looking for. I'm not sure if this setup would be conducive/useful for your flow.Pedro Machado
09/22/2020, 3:20 AMdefault
it can't figure out the dependencies. I tried to add a dependency via set_upstream
and got ValueError: Parameters must be root tasks and can not have upstream dependencies.
I reverted back to my original approach but the thing I don't like about it is that I cannot access the modified parameters through the context. I'd like to use the start/end date to template a task's result location. I'll post another question specifically about accessing upstream inputs when templating the result location.