justabill02/09/2022, 4:12 PM
) support, harnessing the power of concurrency without complex software patterns such as thread pools. The concurrent task runner enables tasks that are blocked on input/output to yield to other tasks, whether your code is synchronous or asynchronous. When running a flow using a task runner that enables concurrent execution, or running many flows across multiple execution environments, you may want to limit the number of certain tasks that can run at the same time. For example, perhaps you want to ensure that, across all of your flows, there are no more than three open connections to your production database at once. You can do so by creating a “prod-db” tag and applying it to all of the tasks that open a connection to that database. Then, you can create a concurrency limit with
. Now, Orion will ensure that no more than 3 task runs with the “prod-db” tag will run at the same time. Previously, this feature was only available in a paid tier of Prefect Cloud, our hosted commercial offering. We’re very happy to move it to the open source domain, furthering our goal of making Orion the most capable workflow orchestration tool ever 💪. There is so much more in this release, including greater flow parameter flexibility, database migrations, CLI refinements, and much more. Read all about it in the release notes! 🚨 Breaking change: If you have previously installed any version of Orion, you'll need to delete existing database with
prefect concurrency-limit create prod-db 3
or stamp it with
to successfully upgrade.
prefect orion database stamp