Hi! Is it possible to increase max scheduled flows...
# ask-communitya
Andrey Vinogradov
04/15/2022, 9:14 AMHi! Is it possible to increase max scheduled flows in prefect? I found “max_scheduled_runs_per_flow” parameter but it seems that is is only available in prefect server not prefect.
a
Anna Geller
04/15/2022, 10:20 AM
That's correct, for Prefect Cloud this is managed globally so you don't need to worry about it. Can you explain the problem you face? Why would you want to adjust it?
Anna Geller
04/15/2022, 10:23 AM
By default, this is set to 10 - "The scheduler periodically queries for flows with active schedules and creates flow runs corresponding to the next 10 scheduled start times of the flow."
but if you want to limit the number of concurrent runs for a specific flow or task, you can use concurrency limits
a
Andrey Vinogradov
04/15/2022, 10:26 AMI use Prefect installed on my server not Prefect Cloud and the problem is that that limit is too small. Is it possible to increase it?
Andrey Vinogradov
04/15/2022, 10:26 AMAs I understand it is possible in Prefect Server not in regular Prefect?
a
Anna Geller
04/15/2022, 10:29 AM
there is Prefect Core (the main package allowing building flows and running them locally)
then there is also Prefect Cloud or Prefect Server that you may use for the backend
Which of those did you install?
• Concurrency limits are only for Prefect Cloud in Prefect 2.0
• max_scheduled_runs_per_flow is only for Prefect Server and this is a global setting allowing you to now overwhelm your Prefect Server
btw if you're just getting started, you may use Prefect 2.0 instead
Anna Geller
04/15/2022, 10:31 AM
limit is too smallwhat are you trying to do? explaining your use case would help immensely
a
Andrey Vinogradov
04/15/2022, 10:33 AMI installed Prefect with
Copy code
pip install prefectAndrey Vinogradov
04/15/2022, 10:33 AMI mean I haven’t installed anything else as the backend
Andrey Vinogradov
04/15/2022, 10:41 AMWe have ml training flow that we run in prefect by schedule but because of this limit we should manually restart schedule for this flow.
Andrey Vinogradov
04/15/2022, 10:41 AMWe want to plan more than 10 runs for this flow in schedule
a
Anna Geller
04/15/2022, 10:48 AM
gotcha, I think this docs page will clarify your confusion: https://docs.prefect.io/orchestration/getting-started/set-up.html
you don't need to worry about the 10 runs. It's just internal detail of the scheduler service, you may run hundreds of flow runs simultaneously
a
Andrey Vinogradov
04/15/2022, 10:49 AMOk thank you!
Andrey Vinogradov
04/15/2022, 1:28 PM@Anna Geller in fact we need run this flow not at the same time but scheduled. So for example we want to plan to run it every 2 hours. Will it work?
k
Kevin Kho
04/15/2022, 1:32 PM
Yes it will, but it’s really recommended to either use Prefect Cloud or Server instead of Prefect Core alone. In core you have to leave the processes running for each Flow. Cloud has 20000 task runs free every month, which is more than enough to get started
171 Views