Stéphan Taljaard
06/18/2021, 1:16 PMevery_30_min = IntervalClock(interval=timedelta(minutes=30), parameter_defaults={"Start date hours delta": 0.75})
daily_schedule_time = pendulum.time(5, 30)
every_day_0530 = IntervalClock(
start_date=pendulum.today(tz=TIMEZONE).at(daily_schedule_time.hour, daily_schedule_time.minute),
interval=timedelta(days=1),
parameter_defaults={"Start DateTime": "yesterday"},
)
schedule = Schedule(clocks=[every_30_min, every_day_0530])
to filter as follows:
• Pull a large chunk data once per day at 05:30
• Pull a small chunk of data every 30 minutes (but not on 05:30)
I think I need a no_filter
, however it seems that applies to all clocks. It will thus remove both my scheduled runs, leaving only the 30minute one?Kevin Kho
06/18/2021, 1:46 PMprefect.context
to get the scheduled time and use that to make replace the Parameter value unfortunately.