:wave: When I schedule a Flow every "12 hours" usi...
# prefect-community
m
👋 When I schedule a Flow every "12 hours" using the
IntervalSchedule
object. Would the flows start at different time or all at the same time ? What I am looking for is to distribute the load on our servers when flows run. And so a new flow should start at a random time and then follow the
IntervalSchedule
constraint.
j
Hi Matt, you could use an IntervalSchedule to achieve this. I believe the interval schedule would run every 12 hrs after registration. You could also try setting two cron schedules and specifying exact times if you wanted to distribute flows very evenly
Copy code
import pendulum
from prefect.schedules import Schedule, clocks

schedule = Schedule(clocks=clocks.CronClock(
                 "0 0 * * *",
                 start_date=pendulum.now("US/Central")
)
schedule2 = Schedule(clocks=clocks.CronClock(
                 "0 12 * * *",
                 start_date=pendulum.now("US/Central")
)
m
Yes but Prefect makes things start at the hour. So when I look at my servers, they are busy from when the hour starts to 10min later. For example, it's busy from 1 pm to 1:10 pm and idle from 1:10 pm to 2 pm What I would like is Prefect to schedule my IntervalSchedule jobs totally randomly at first and then stick to it.
The I get random OOM errors because 10 jobs started exactly at 1 pm
This is for example a flow I have
Copy code
with Flow(
    "flow_name",
    schedule=IntervalSchedule(
        interval=timedelta(hours=12),
    ),
) as flow:
    ...
Does that make sense to you ?
Hi @James Sopkin Just want to make sure you saw my late reply 🙏
j
@Matt Delacour Apologies for the late reply. Prefect was out on a company retreat last week. Currently there is no feature for random times for interval schedules. You could try using filters to set specific times see here in the filters section -> https://docs.prefect.io/core/concepts/schedules.html#clocks
So try spacing the flows out with different starting times to reduce the load on your infrastructure
m
So try spacing the flows out with different starting times to reduce the load on your infrastructure
Yes I guess, I will go with a random function for the start time to schedule the jobs. Might want to do it deterministically (eg: using the name of the flow)
👍 1