Anna Geller

08/17/2022, 11:08 PM
Hi everyone! 😛refect-duck: We are excited to announce the 2.1 release! 😛refect2: :cloud2: See the description below 👇 for a list of new (highly requested!) features. We don't even mention here all the tiny improvements and bug fixes you can read more about in the Discourse announcement and the release notes. Build Deployments in Python The
prefect deployment build
CLI provides a simple way of creating deployments, and it has been designed for reliable engineering workflows that integrate well with CI/CD systems such as GitHub Actions (tutorials and recipes follow soon!). However, some users requested an additional feature to also allow deployment creation purely using Python. See the Discourse announcement with an example of how you can use that deployment pattern. The documentation has been extended accordingly. --- Simplified Agents & Work Queues Agents and work queues give you control over where and how flow runs are executed. Now, creating an agent (and corresponding work queue) is even easier. Why? • Work queues now operate strictly by name, not by matching tags. Tags remain an important concept for organizing your work based on projects and filtering your deployments and runs in the UI. • Deployments, and the flow runs they generate, are explicitly linked to a single work queue - this means that deployments map 1:1 to work queues rather than via a many-to-many relationship. • The work queue is automatically created whenever a deployment references it - this means that you no longer have to worry about creating a work queue manually; assign a new work queue name to your deployment when you need it, and just start a new agent polling from the same work queue name. Prefect will handle work-queue creation under the hood for you. Here is what this looks like in practice:
prefect deployment build -n mydeployment -q prod -o my.yaml
prefect deployment apply my.yaml

prefect deployment run myfunc/mydeployment 

prefect agent start -q prod
This “prod” work queue will pick up the run since it matches the queue
-q prod
specified on this deployment. Default work queue If you forget a work queue name on your deployment, Prefect will automatically assign a “default” work-queue name. This way, the runs generated by such deployments can be picked up by an agent you can start using:
prefect agent start -q default
Benefits of the work queue changes Most users will not have to interact directly with work queues at all – you only need to interact with deployments and agents. You can use work queues only once you need a more advanced configuration, such as using concurrency limits. Work queue changes are backward compatible! The changes are fully backward compatible. See the documentation to learn more. --- New Collections We announced those here already, but to list those again, here are recently released Prefect Collections: • prefect-mondayprefect-databricksprefect-fugue As always, to upgrade to the latest version:
pip install prefect -U
Cloud 2.0 is already up to date! :cloud2: Happy engineering! :marvin-duck:
🎉 8
😀 5
:cloud2: 11
:prefect2: 10
:gratitude-thank-you: 22
🚀 25
:fugue: 8
:databricks: 7