Prefect Schedules + Filters: I have two schedules...
# ask-community
s
Prefect Schedules + Filters: I have two schedules on the same flow: • Pull a small chunk of data every 30 minutes • Pull a large chunk data once per day at 05:30 On 05:30, two runs will be scheduled - one for the small chunk and one for the large chunk. However, since there is a small overlap in time, duplicate data will be pulled. Is there a way to edit the following schedule
Copy code
every_30_min = IntervalClock(interval=timedelta(minutes=30), parameter_defaults={"Start date hours delta": 0.75})
daily_schedule_time = pendulum.time(5, 30)
every_day_0530 = IntervalClock(
    start_date=pendulum.today(tz=TIMEZONE).at(daily_schedule_time.hour, daily_schedule_time.minute),
    interval=timedelta(days=1),
    parameter_defaults={"Start DateTime": "yesterday"},
)

schedule = Schedule(clocks=[every_30_min, every_day_0530])
to filter as follows: • Pull a large chunk data once per day at 05:30 • Pull a small chunk of data every 30 minutes (but not on 05:30) I think I need a
no_filter
, however it seems that applies to all clocks. It will thus remove both my scheduled runs, leaving only the 30minute one?
k
Hey @Stéphan Taljaard, this is a really good question. Will ask the team
hattip 1
Ok so the best answer for this is you combine and into one clock and probably use the
prefect.context
to get the scheduled time and use that to make replace the Parameter value unfortunately.