https://prefect.io
Join Slack
I have scheduled a job to run daily at 7:am UTC and it takes current_date, from_date and to_date as...
# ask-community
s
I have scheduled a job to run daily at 7:am UTC and it takes current_date, from_date and to_date as parameter but seems like it is not updating and keeps running the same date when i registered the flow?
k
Hey @Sarita Patel, it seems like 
current_date
, 
from_date
and 
to_date
are being evaluated when you register which is the default. Instead you can do something like:
Copy code
@task
def get_day(day):
    if day == "today":
        return datetime.datetime.today()

with Flow(...) as flow:
    day = Parameter("day", default="today")
    day = get_day(day)
s
Copy code
from_date = Parameter('from_date', default=(datetime.now() - timedelta(days=2)).date().isoformat())
to_date = Parameter('to_date', default=(datetime.now() - timedelta(days=1)).date().isoformat())
curr_date = Parameter('curr_date', default=(datetime.now()).date().isoformat())
I register the flow yesterday:
and these were the parameter
But for today's schedule it should be updated to current_date= 2nd Dec
from_date = 30th Dec to_date = 1st Dec
k
You can’t put a callable as a default value. It gets evaluated immediately during registration time
You need to use a task to defer the execution to run time
s
ahh.. I see. Earlier it worked, did something changed recently?
Also, can you share an example where I can defer the execution of Parameter assignment at run time?
k
My code snippet above is the recommended approach. I think callables in Parameter defaults was never supported. What is your storage? There was a related change though, but in general the callables was not something explicitly supported even if it may have worked. It was a side effect of other issues
s
I see that this change merged on Oct 7 and since Oct 7 my flow parameters were not picking up the current date. So, now I get that callable are not allowed in Parameters, however, can I use Prefect Context as default value for Parameters?
k
No unless you call it inside a task. I don’t think context will be populated during registration time. It will also fix a default value for the Parameter if the value exists. The only way to introduce some dynamicism is through using tasks.
👍 1
s
Got it. Thanks for your inputs
k
Of course!
40 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by