👋 Hi there Prefect Cloud 2 :cloud2: users! I’m Nathaniel from the Product team at Prefect. We wanted to share a bit about a new addition to Prefect Cloud 2, the Flow Run Retention Period setting. With the Flow Run Retention Period setting, those with additional compliance or analytics needs can now choose to retain historical data about their flow runs in Prefect Cloud for an extended period of time. This flexibility in retention period allows those writing, running, or depending on flows to investigate the final state, logs, and run pattern of flows and their tasks – long after they complete. The retention period can also support you in maintaining compliance with minimum retention policies that your organization may have, alongside providing an expanded window into the history of your flow runs. What does this mean for you? All existing Prefect Cloud 2 users and organizations automatically receive a 180 day retention period setting, the same as our Enterprise tier. This means that after a flow run ends, its details are available for 180 days before being removed. Note 📝: This setting does not impact the existing ability to manually delete flow runs via the Cloud UI or CLI. Flow runs can still manually be deleted, and remain deleted – regardless of the retention period setting. Moving forward, new individual users of Prefect Cloud 2 in the Starter tier will have flows runs maintained for 7 days after completion. Those who have upgraded to the Growth or Enterprise tiers will have a 30 and 180 day flow run retention period respectively by default. Additionally, if you or your organization need a tailored retention period, our Sales team or your account executive can help with customized periods above and beyond these defaults.

Hi everyone! 👋 We are excited to announce the 2.3.0 release! 😛refect2: :cloud2: It includes some highly requested new features such as: • first-class support for flows stored in Docker images and in read-only GitHub storage blocks • context manager

disable_run_logger

allowing to disable Prefect logger for testing tasks and flows • new task run UI page • several fixes and enhancements -- many of those include improvements to Deployments. As always, thanks to all contributors and everyone reporting issues on GitHub! 💙 For a detailed overview of changes, see the blog post for the release. Happy Engineering! 😛refect:

Hey all! 👋 Prefect 2.3.1 was just released! It contains a number of bug fixes and enhancements. And we released a prefect-jupyter collection for running Jupyter notebooks in flows! 🪐 Enhancements • Add sync compatibility to

run

for all infrastructure types — #6654 • Update Docker container name collision log to

INFO

level for clarity — #6657 • Refactor block documents queries for speed ⚡️ — #6645 • Update block CLI to match standard styling — #6679 Fixes • Add

git

to the Prefect image — #6653 • Update Docker container runs to be robust to container removal — #6656 • Fix parsing of

PREFECT_TEST_MODE

in

PrefectBaseModel

— #6647 • Fix handling of

.prefectignore

paths on Windows — #6680 Contributors • @Mars made their first contribution — #6639 • @pdashk made their first contribution — #6640

pip install -U prefect

to upgrade! 🎉

Hi everyone! Prefect 2.3.2 😛refect2: was released just now. It primarily includes fixes and enhancements to the UI and documentation. As always, release notes on Discourse and GitHub provide a full list of changes. Shoutout to lennertvandevelde for becoming a new Prefect contributor! 🎉 And thanks to everyone raising issues on GitHub. Have a great rest of the week! 😛refect-duck:

Hello everyone! Yesterday the Prefect repository crossed the 10,000 stars threshold. Although we don’t measure our impact solely by GitHub stars, it is a great reason to take a moment and reflect on the community that we have all helped to build over the past 3-4 years. Fun fact: @Dustin Ngo was the first person to star the repo after myself and @Jeremiah, and is now a member of the Prefect Team! It is thanks to your contributions, feedback and usage that so many new people every day are adopting Prefect. I remember when we first open sourced, we agreed that 100 stars in the first 24 hours would be a success. When we blew past 1k we knew we were onto something special. Considering how much Prefect has changed since that time (due in massive part to the feedback and encouragement from this community), I’m nothing but optimistic and excited about the things we have in store for the next phase…

Hi all! We have some upcoming community meetups in DC and Boston, and our first ever community coworking day in DC coming up. https://prefect-community.slack.com/archives/C036FRC4KMW/p1663072997030719 Check out the #events channel to learn more and stay updated on all of the Prefect happenings!

Hi everyone! 👋 We are excited to announce the 2.4.0 release! 😛refect2: It includes some highly requested new features, including: •

ECSTask

infrastructure block - shoutout to @Michael Adkins for making it so easy to use and applicable to a variety of use cases (Fargate, EC2, Spot instances, and tons of flexible configuration options). Note that we are currently releasing this as an experimental block ⚠️ - it's ready for early feedback, but if you plan to use it in production, be aware that this block may change without notice in the next couple of releases, • Support for setting custom parameter values directly from the deployment build CLI • New CLI for interacting with and setting custom timezone-dependent schedules for your deployments incl. the ability to pause or resume a schedule directly from the CLI --- On top of that, this release includes some really cool enhancements and important fixes, a.o.: • improved query performance and protecting against race conditions when disabling and reenabling schedules, • fixes to blocks and orchestration metadata context • several documentation improvements, including the brand new catalog of Prefect Recipes 🧑‍🍳 As always, to upgrade:

pip install prefect -U

Thanks to all contributors and everyone reporting issues on GitHub! For a detailed overview of changes, see the release notes. Happy Engineering! 😛refect:

Hi everyone! We've just released Prefect 2.4.1 😛refect2: :cloud2: To read more about all the cool enhancements and fixes, check out our release post. Thanks so much to all contributors (there were many!) 💙

Hi again, Release 2.4.2 has just shipped with fixes to blocks in 2.4.1, see the release notes for details. Thanks so much to our users who found and helped fix these issues!

Hi everyone, happy Thursday! 😛refect-duck: We've just shipped Prefect 2.4.3. 😛refect2: This release includes performance enhancements, CLI and documentation improvements, and numerous fixes - full release notes available here. :cloud2: In addition, there are two notable integrations: 1. New `prefect-hex` collection — adding in #6974 2. New `CloudRunJob` infrastructure block added to

prefect-gcp

in the PR PrefectHQ/prefect-gcp#48 As always, thanks to everyone reporting issues on GitHub, and kudos to two new contributors: • @Hongbo Miao - in #6956 • @hateyouinfinity - in #6955 Happy Engineering! 😛refect:

A quick update to the above: we've just pushed two patches fixing a block protection issue. The latest 2.4.5 release restores the ability for a client and server to have a different version. New safeguards for that will be added in the future. To learn more, check the release notes here. Big thanks to @Michael Adkins, @alex, @nicholas, @Jake Kaplan and @Evan Sutherland for shipping those patches so quickly ⭐

Hi all 👋, Will here from the Product team at Prefect. Starting today, we are entering our second phase on our mission to coordinate the world’s dataflow. We are making Prefect Cloud 2.0’s commercial and enterprise features available to self-serve users with no limits. :cloud2: We recently announced Prefect Cloud 2.0 with an expansive free tier that permits unlimited usage. The response to its GA launch was overwhelming: last week alone, we processed over 6 million tasks, representing roughly 20% as much work as Cloud 1.0 currently handles. To cement this momentum, today we’re releasing a new suite of commercial and enterprise features as the second phase of Prefect Cloud 2.0’s rollout. 🎉 Prefect Cloud 2.0 introduces a new, self-serve Organization tier that includes critical collaboration features for large teams: organizations, service accounts, team management, roles, and more. In conjunction, we’ve also removed all caps from our self-serve tiers; our engineering-led sales effort means you don’t need to talk to us to use Prefect in production. Organizations will also receive some features previously reserved for our enterprise users, such as an audit log. 🏢 For a full breakdown of our new unlimited Prefect Cloud self-serve tiers, check out prefect.io/pricing.

Hey there! 👋 Nathaniel from Prefect here! We’re excited to announce the addition of custom workspace roles (custom RBAC) to the Prefect Cloud 2 :cloud2: enterprise plan. Organizations supercharge an account admin’s flexibility in ensuring that users of Prefect Cloud 2 have access to the right Prefect functionality, in the right workspaces. Custom workspace roles are composed of granular permissions to perform certain activities within a workspace, so people in your organization who might not need all of Prefect’s functionality can be given access to what helps them get work done. Custom workspace roles can inherit permissions from a default workspace role, enabling you to tweak beyond the default roles to meet your organization’s needs. Custom workspace roles can also be created independent of Prefect’s default workspace roles. This option gives workspace admins full control of user access to workspace functionality, including determining access to new functionality as it is released. If you’re interested in learning more about custom workspace roles in Prefect Cloud 2, check out the documentation here, or contact our sales team via sales@prefect.io to see if the enterprise plan might be a good fit for your organization.

Hi everyone! 👋 We've just released Prefect 2.5 😛refect2: :cloud2: with several highly requested features, including: •

run_deployment

utility, allowing you to trigger flow runs from deployments, wait for their completion, schedule runs for later (even from a custom application), coordinate parametrized dataflows across teams, and run each part of your workflow in a separate infrastructure (e.g., a Docker container or a Kubernetes job). Huge thanks to @Dustin Ngo and @Michael Adkins for building this so quickly based on user feedback • improvements to serverless deployments, including a.o. Google Cloud Run infrastructure block, and ECS customizations • batch deletion of flow runs from the UI (and more UI features coming soon!) - kudos to @Marichka Offen Check the release post to find out more. Thanks to everyone reporting issues on GitHub and to @ale and @Stéphan Taljaard for contributing.

Hi everyone!👋 I made a short video to tell you about some of the updates we just added to the Prefect Cloud :cloud2: UI and which are coming soon to OSS 😛refect2:: • Saved views - you can now save a set of state, flow, deployment and tag filters to customize how you see the flow runs page • Multi-delete - you can select multiple flow runs and delete them from the UI • View active runs - we made it easier to see which runs are taking up concurrency on your work queues We hope they make using the UI easier. We're working on more updates so please let us know if there's anything else you would like to see or do in the UI 🚀.

See you all at Coalesce :dbt: ! https://prefect-community.slack.com/archives/C036FRC4KMW/p1665603566629469

Hi everyone! We've just released Prefect 2.6 😛refect2: with the first-class result configuration, new

allow_failure

annotation, dynamic work-queue patterns for easier configuration of concurrency limits, and more. Check out the release blog post and release notes for more details. Happy Engineering! :marvin-duck:

Hi everyone! 👋 We've just created a new channel #prefect-dbt for all questions, comments, blogs, collaboration and fun posts related to DBT :dbt: If you're at Coalesce this week, pop in the channel and post about interesting learnings, pictures of the event, or questions about DBT.

Hi everyone! Prefect 2.6.3 is out 😛refect2: :cloud2: It includes some major enhancements, including support for compound recurrence rule schedules and private GitHub repositories. There are also fixes and enhancements for the new results feature; notably, we now support reducing memory consumption by optionally always retrieving results from storage. As always, the highlights above are a small subset of the changes. See the release notes for everything else. Happy Hacktoberfest 👨‍💻, happy Coalesce :dbt: and happy engineering!😛refect:

Hi everyone, happy Thursday! Two releases in a single week? Yup, the Prefect team is shipping it! Prefect 2.6.4 😛refect2: :cloud2: is live and provides great news for serverless fans! :aws:

ECSTask

infrastructure block is production-ready! We've got great feedback from the community that we used to improve the UX, and now this block is no longer experimental! Thanks to everyone who shared feedback and contributed feature requests on the prefect-aws repository. :azure:

AzureContainerInstanceJob

infrastructure block for running containers on Azure is now released as part of the prefect-azure collection - @Ryan Peden will share a blog post demonstrating how to use this infrastructure block very soon. One of the coolest characteristics of Azure Container Instances is that it allows you to run serverless containers with GPUs - extremely helpful for all data scientists using Prefect and Azure to run GPU-powered ML workflows. 🔑 a new key-value store collection contributed by @Michael Adkins! Apart from collections, this release includes bug fixes and enhancements to the core library. See the release notes for details. If you haven't checked out 2.6 yet, you should now! We launched first-class configuration of results in 2.6.0, and every release since includes additional improvements to result handling. Happy Engineering!

Join us live now to learn about event-driven workflows live at dbt Coalesce! (You’ll need to register free first here) https://attendees.bizzabo.com/396530/agenda/activity/967496

👋 Howdy everyone! We normally keep track of Prefect livestreams and IRL events in the #events channel. However, as you may have noticed, it can be really hard to keep track of upcoming events that you care about since there are now so many! Today we're releasing the Prefect Community Calendar, available at prefec.tv/calendar. This can make it easy to check out information about single events, or to subscribe to the entire calendar. Check it out, subscribe, and we look forward to seeing you all at a Prefect livestream, meetup or other IRL event in the future!

Hi everyone! Prefect 2.6.5 😛refect2: :cloud2: is here! It adds the highly requested manual retry from failure directly from the UI. The image below shows what it looks like from the UI. Note that, by default, Prefect doesn’t persist results unless you leverage a feature that needs it (such as automatic retries or caching). This means that when you pass data between tasks or subflows and you want to leverage manual retries from the UI, you need to enable result persistence which can be toggled with the following setting:

prefect config set PREFECT_RESULTS_PERSIST_BY_DEFAULT=true

. Other enhancements include: • support for Python 3.11 😛ython: • a new API route for work queue status check (UI work queue health check page is on the roadmap, as well) ☑️ • documentation showing how you can configure SSO for Prefect Cloud directly from the UI • two new 📦 Prefect Collections: prefect-docker and prefect-census For more info about all fixes and enhancements, see the release notes. As always, to upgrade:

pip install prefect -U

Have a great rest of the week!

Hi everyone! 👋 😒lack: Today we are adding 11 new channels to our Slack Community to open up conversations and discussions relating not only Prefect-specific topics but data engineering and data science. We want this Community to be a place where you can collaborate and get help with your data regardless of the stack or tool you are using. We can all share tips, tricks, and learn together. We have new cloud-specific channels for #prefect-aws, #prefect-gcp, #prefect-azure Channels dedicated to #prefect-kubernetes, #prefect-docker, #prefect-dbt, and #prefect-cloud New channels for #prefect-integrations, #prefect-recipes and #prefect-getting-started that will have live stream events for Build-a-Block or Code with Marvin. Lastly, we are so grateful for all of you in our Community that we wanted to create a #gratitude channel where we can all share our thanks for one another. So make sure to check out the new channels, join the conversations, and look for new events and announcements. tl;dr - 11 new channels in slack and lots of new events or watch the video 😉 📺 https://www.loom.com/share/5ad9da90d6ab4beeadcbd7de6ef7a527

Hi Prefect Community! Last week, we became aware of vulnerabilities in OpenSSL and began assessing potential impact to our internal infrastructure and container images that we publish for our open source projects. Prefect reviewed the OpenSSL advisory and the details of the two vulnerabilities (CVE-2022-3786 and CVE-2022-3602), which impact OpenSSL versions 3.0 and above, under certain conditions during certificate validation. Prefect uses a Google Cloud Load Balancer to process all inbound traffic for both versions of Prefect Cloud, including cryptography, and we continuously monitor Google’s security advisories, including their corresponding OpenSSL advisory. In one instance, we identified a vulnerable version of OpenSSL in a container image provided by an upstream vendor and confirmed that the application does not link to the libraries (libssl or libcrypto), and so is not affected by this issue.. Additionally, the service is accessible only to Prefect employees as a result of network-level restrictions. No portion of our infrastructure was affected by this issue, and we reaffirm our commitment to provide our customers with a timely security update if we discover otherwise. 🔐 The most recent versions of the container images that Prefect publishes for our open source projects (Prefect 1.0, Prefect Server 1.0, and Prefect 2.0) include an older version of OpenSSL, and we do not believe they are affected. ✅ We will continue to monitor the situation as it develops and update this page accordingly. Don't hesitate to contact the Customer Success team or ask me questions directly! :highfive:

Hi everyone! We’ve just released 😛refect2: :cloud2: Prefect 2.6.6, which includes: • work queue health status page • scheduler performance improvements • documentation for our versioning scheme (explaining why this is a 2.6.6 release rather than 2.7) • new Prefect Collections for GitLab storage, Alert notifications, and Fivetran data ingestion The image below shows the highly informative UI page with one healthy and one unhealthy work queue. As always, the release notes provide more details. Happy Engineering! :marvin-duck:

Hey there! 👋 Nathaniel from Prefect here! Today, we’re excited to introduce new compliance and transparency functionality for organizations with the addition of audit log in Prefect Cloud 2. Audit log provides organizations a chronological record of activities performed by Prefect Cloud users, allowing you to monitor detailed Prefect Cloud actions for security and compliance purposes. In conjunction with appropriate tools and procedures, audit log can assist in detecting potential security violations, and investigating application errors. Audit log enables you to identify who took action, and on what items within your Prefect Cloud organization (when relevant). Starting today, audit log can be used to identify changes in access to workspaces, user login activity, API key generation, and service account creation, just to name a few. All organizations have access to 72 hours of audit log retention, meaning that you can get access to audit log self service by creating an organization today. Those on the Enterprise tier have access to 180 days of audit log retention by default. Additionally, if you or your organization need a tailored retention period, our team at sales@prefect.io can help with customized periods above and beyond these defaults.

Hi everyone! 😛refect2: :cloud2: Prefect 2.6.7 is out and adds: • timeouts for task runs, • colored logging, • obfuscation of secret settings, • dark mode for documentation, • and more! It also includes an incredible amount of usability updates in the UI (e.g., when you navigate to a subflow run, UI will now redirect directly to the logs tab and will additionally display a link to the parent flow run that triggered that child flow run), performance improvements to the scheduler, and lots of bug fixes. We also added a new

prefect-hightouch

collection for Hightouch to help you reverse your ETL 😎 As always, you can see the release notes for more details.