Hey! Can anyone confirm for me that if a schedule contains more than 10 (concurrent) runs, this is limited to 10 runs only. My schedule contains 20 runs (with different parameters for a flow) where many start at the same time. However, when I deploy this schedule to Prefect only 10 runs are scheduled instead of the 20. This is on Prefect's website: There is no limit to the number of flow runs you can create, even if they start at the exact same time with the exact same parameters: Prefect supports unlimited concurrent execution. This does not seem the case for me.

Hey @Fabrice Toussaint -- this may be a limitation of our scheduler which schedules 10 runs in the future at a time. I'm not sure it will actually stop your flows from running on time though, I will investigate that.

Hi @Zanie, thanks for the quick reply! I split it up into multiple flows and make use of the StartFlowRun task from Prefect as this gives me more flexibility. However, still only 10 flows were run in the previous setup even though they were scheduled (or they did not show in the UI?).

If they ran they would show up in the UI -- that's peculiar. It's possible they missed being scheduled entirely because of how the scheduler queries for flows to schedule. We'll definitely be looking at improving this soon.

@Zanie for some reason even with the scheduling the StartFlowRun only the first schedule is actually executed. So I have a flow with StartFlowRuns and a flow which is triggered by that flow. The flow with StartFlowRuns keeps executing and scheduling but none of these tasks actually trigger the other flow to run. Any idea why this happening?

It sounds likely that you're encountering the default idempotency key value

- idempotency_key (str, optional): a unique idempotency key for scheduling the
    flow run. Duplicate flow runs with the same idempotency key will only create
    a single flow run. This is useful for ensuring that only one run is created
    if this task is retried. If not provided, defaults to the active `task_run_id`.

An example can be seen below:

with Flow as flow:
   StartFlowRun().run(flow_name='flowname', project_name='projcctname',
                        parameters={}) 
   StartFlowRun().run(flow_name='flowname', project_name='projcctname',
                        parameters={})

with this schedule

schedule = Schedule(clocks=[clocks.CronClock('0 */2 * * *')])

Oh, don't call

.run()

directly

How do I get it running then or does it start running as in your example?

So Prefect 'Task' objects are both callable and have a

run

method.

I will remember this, thank you very much and sorry for the mistake 😛

That's alright! It's a bit tricky haha