Prefect 2.8.4 has been released! We are excited to celebrate many first-time contributors in this release 🎉: • @samdyzon upgraded the Azure filesystem block in #7513 👏 • @jcorrado76 added to the deployment CLI in #8603 🥳 • @mjschock added a docs fix in #8660 🙌 • @scharlottej13 updated Dask's logo in #8669 🎊 Thank you for your contributions and we look forward to your continued involvement in the Prefect community! 💪 Additionally, this release brings several other exciting changes, including added support for YAML config strings to

KubernetesClusterConfig

, graceful SIGTERM handling in Prefect Server, and much more. See the release notes for a complete list of changes and bug fixes. And for Google Cloud users, we also released a new version of the prefect-gcp collection!

Hello all! I wanted to share a UI update that we've added the ability to use markdown for your deployment descriptions. You can see it now in cloud and should be able to see it in OSS after our next release.

👋 Happy Wednesday!! 🐫 ℹ️ I'd like to take a moment to share some work we've done to surface flows from collections in the UI. If you hop into the

Flows

tab of your Prefect Cloud UI, you should notice a new button

Add from Collections

. When you click it, you'll see a variety of different flows that we've defined in our collections ecosystem. For each flow, you'll see an example of how that flow can be used and a link to its docs. Each example is taken directly from the docstring of that flow, so for example if you click on

run-dbt-cloud-job

, you'll find a nice markdown render of this docstring. The nice thing about this is that if you've developed a nice way of a using a flow from a given collection, you can open a PR to that collection to add your example to the flow's docstring - and after we review and merge your PR, it will appear here in the UI! 🗣️ If you have any feedback, questions, or if you want to learn more about how you can include a flow of yours here... reach out in #prefect-integrations or #prefect-contributors and we'll help you along marvin Happy Engineering!

Prefect 2.8.5 has been released! We're excited to celebrate two first-time contributors in this release! 🎉 • @Zesky665 corrected a typo in the docs in #8692 🔍 • @predatorprasad fixed a CLI help message in #8755 ❓ Thanks for your contributions, and thank you for being a part of the Prefect community! This release includes several fixes and enhancements, including: • Faster deployment deletion • Enhanced CLI login reliability • A fix for Kubernetes job timeout errors • And more! See the release notes for full details.

Register to attend our upcoming livestream with Ray + joint user LiveEO!

We just announced

langchain-prefect

, a plugin for building with LLMs at scale. Read more here or check it out on GitHub. If you have any questions come join us at #prefect-ai!

Prefect 2.8.6 has been released! Prefect Runtime For Context Access Many users of Prefect run their flows in highly dynamic environments; because of this it can be incredibly useful to access information about the current flow run or deployment run outside of a flow function for configuration purposes. For example, if we are running a Prefect deployment within a larger Dask cluster, we might want to use each flow run id as the Dask client name for easier searching of the scheduler logs. Prefect now offers a user-friendly way of accessing this information through the

prefect.runtime

namespace:

from prefect.runtime import flow_run
from prefect import flow
from prefect_dask.task_runners import DaskTaskRunner

@flow(task_runner=DaskTaskRunner(client_kwargs = {"name": flow_run.id}))
def my_flow():
    ...

This will create a Dask client whose name mirrors the flow run ID. Similarly, you can use

prefect.runtime

to access parameters that were passed to this deployment run via

prefect.runtime.deployment.parameters

. Note that all of these attributes will be empty if they are not available. See #8790 for details. A few other key enhancements and fixes: Enhancements • Add deployment ID support to

run_deployment

🪪 — #7958 • Disable Postgres JIT for performance improvements 👟 — #8804 Fixes • Fix blocking file read in async method

Deployment.load_from_yaml

🛠️ — #8798 • Allow tasks and flows to make redundant transitions such as

RUNNING

->

RUNNING

🏃‍♂️ 🏃‍♀️ — #8802 Documentation • Update workspace roles table to emphasize differences between roles — #8787 • Add Webhook block docs 🪝 — #8773 • Update info on Ray's support for hardware and software ray— #8811 Helm chart • Helm charts are now automatically published on each Prefect release kubernetes — #8776 Contributors • @devanshdoshi9 See the release notes for a full list of changes!

Prefect 2.8.7 is here! Our latest release introduces improved concurrency handling, flow run graph enhancements, and result artifacts, elevating the Prefect experience. We’ve made major improvements to the Prefect engine, supporting mixed asynchronous and synchronous code, closing long-standing bugs, and ensuring reliability when running flows. ‏‏‎ ‎ Key concurrency and engine reliability enhancements • Dedicated worker thread for flow and task orchestration. • Synchronous flows run on main thread, resolving issues like #5991. • Asynchronous flows have separate event loop from Prefect engine. • Flow timeouts enforced with signals for effective interruption of long-running calls. • Async flows callable from sync flows; async tasks usable as upstream dependencies for sync tasks. • Deadlocks prevented when waiting for many sleeping tasks or having large numbers of synchronous tasks. • Debug mode enables verbose logging from Prefect concurrency internals. • API limits to 100 concurrent requests with SQLite backend to avoid database contention. • Task input resolution no longer uses worker threads, preventing deadlocks with numerous upstream inputs. ‏‏‎ ‎ Results tab on flow run pages The Prefect UI now renders information about your flow run and task run results! This view provides a visual representation of the output of your tasks and flows and, when possible, provides links to results persisted using any of our storage blocks. To see this in your UI, run any flow and navigate to the run page; from there you’ll see a new tab, “Results”. See the attached images for examples. ‏ Flow run graph We heard that people loved the simplicity and sleekness of the timeline on the flow run page, but valued the radar graph’s ability to traverse between flow runs and subflows runs. This release introduces the ability to expand and collapse subflow runs within the timeline. With these enhancements, the flow run timeline has now evolved into a general purpose flow run graph, with the ability to render thousands of nodes and edges performantly. The radar graph has been retired. You can now observe and explore your flow runs even more quickly and easily in a single flow run graph! Examples are attached below. Other Notable Changes ‏‏‎ ‎ Enhancements • Reverse option for flow run logs CLI (#8625). • Full run history in UI (#8885). • Improved SQLite sync mode performance (#8071). • Support for variadic keyword arguments in Task.map (#8188). • Add deployment.name and deployment.version to prefect.runtime (#8864). ‏‏‎ ‎ Fixes • Resolved issues with race conditions (#8886), configuration (#8872), and recursion limits (#8827). • Fixed state message display in flow run metadata panel (#8885). • Fixed prefect dev start failure (#8850). • Resolved image pull errors with certain cloud providers by changing Docker build configuration(#8889). ‏‏‎ ‎ For additional information, view the complete release notes.

👋 hey all! We're excited to announce a new Support tier dedicated to our Open Source and Self-Serve customers! Starting today, Prefect Cloud self-serve users or Prefect open source users can purchase Standard support by sending an email to

<mailto:help@prefect.io|help@prefect.io>

or meeting with one of our product advocates directly. The price will be a flat fee of $500/mo, billed annually via a separate invoice. This plan includes: • Access to our Dedicated Support Team • 9a-5p CST SLA • 1 Hour Initial Technical Sync Our talented Solution Engineering team is looking forward to delivering incredible support for our open source and self-serve developers! To get support for your open source deployment or self-serve Prefect Cloud account, reach out to

<mailto:help@prefect.io|help@prefect.io>

today.

Prefect 2.9.0 is here with Artifacts, configurable result storage keys, and runtime enhancements! 🦜 Track and manage artifacts 📜 Most workflows produce or update an artifact of some kind, whether it’s a table, a file, or a model. With Prefect Artifacts, you can track changes to these outputs and richly display them in the UI as tables, markdown, and links. ‏‏‎ ‎ Artifacts may be associated with a particular task run, flow run, or even exist outside a flow run context, enabling you to not only observe your flows, but the objects that they interact with as well - see the first screenshot attached below. A variety of artifact types are available. To create an artifact that produces a table, for example, you can use the

create_table_artifact()

function: ‏‏‎ ‎

from prefect import task, flow
from prefect.artifacts import create_table_artifact

@task
def my_table_task():
    table_data = [
        {"id": 0, "name": "Dublin", "lat": 53.3498, "lon": -6.2603,},
        {"id": 1, "name": "London", "lat": 51.5074, "lon": -0.1278,},
        {"id": 2, "name": "New York", "lat": 40.7128, "lon": -74.0060,},
        {"id": 3, "name": "Oslo", "lat": 59.9139, "lon": 10.7522,},
        {"id": 4, "name": "Paris", "lat": 48.8566, "lon": 2.3522,},
        {"id": 5, "name": "Rome", "lat": 41.9028, "lon": 12.4964,},
        {"id": 6, "name": "Tokyo", "lat": 35.6895, "lon": 139.6917,},
        {"id": 7, "name": "Vancouver", "lat": 49.2827, "lon": -123.1207,}
    ]

    return create_table_artifact(
        key="cities-table",
        table=table_data,
        description="A table of cities and their coordinates",
    )

@flow
def my_flow():
    table = my_table_task()
    return table

if __name__ == "__main__":
    my_flow()

You can view your artifacts in the Artifacts page of the Prefect UI, easily search the data in your new table artifact, and toggle between a rendered and raw version of your data - see the second screenshot attached below. • See the documentation to learn more about Artifacts! ‏‏‎ ‎ Configure result storage keys 🔑 When persisting results, Prefect stores data at a unique, randomly-generated path. While this is convenient for ensuring the result is never overwritten, it limits organization of result files. In this release, we’ve added configuration of result storage keys, which gives you control over the result file path. Result storage keys can be dynamically formatted with access to all of the modules in

prefect.runtime

and the run’s

parameters

. ‏‏‎ ‎ For example, you can name each result to correspond to the flow run that produced it and a parameter it received:

from prefect import flow, task

@flow()
def my_flow():
    hello_world()
    hello_world(name="foo")
    hello_world(name="bar")

@task(
    persist_result=True,
    result_storage_key="hello__{flow_run.name}__{parameters[name]}.json",
)
def hello_world(name: str = "world"):
    return f"hello {name}"

my_flow()

‏‏‎ ‎ Which will persist three result files in the storage directory:

$ ls ~/.prefect/storage | grep "hello__" 
hello__rousing-mushroom__bar.json
hello__rousing-mushroom__foo.json
hello__rousing-mushroom__world.json

‏‏‎ ‏‏‎ ‎‎ See the documentation for more information. ‏‏‎ ‎ ‏‏‎ ‎ Expanded

prefect.runtime

‏‏‎ ‎ The

prefect.runtime

module is now the preferred way to access information about the current run. In this release, we’ve added the following attributes: •

prefect.runtime.task_run.id

•

prefect.runtime.task_run.name

•

prefect.runtime.task_run.task_name

•

prefect.runtime.task_run.tags

•

prefect.runtime.task_run.parameters

•

prefect.runtime.flow_run.name

•

prefect.runtime.flow_run.flow_name

•

prefect.runtime.flow_run.parameters

‏‏‎ ‎ See the documentation for more information! Contributors We have several first-time contributors in this release. Let’s give them a round of applause! 👏 ‏‏‎ ‎ • @andreadistefano made their first contribution in #8942 • @knl made their first contribution in #8974 • @thomas-te made their first contribution in #8959 Here are a few key enhancements and fixes: ‏‏‎ ‎ Enhancements • 🔢 Add unique integers to worker thread names for inspection - #8908 • 🔁 Add support to

JSONSerializer

for serialization of exceptions so they are persisted even on failure - #8922 • 🗜️ Add Gzip middleware to the UI and API FastAPI apps for compressing responses - #8931 • 🔎 Update the runtime to detect flow run information from task run contexts — #8951 ‏‏‎ ‎ Fixes • 🛠️ Fix imports in copytree backport for Python 3.7 - #8925 • 🔄 Retry on sqlite operational errors - #8950 • ⏱️ Add 30 second timeout to shutdown of the log worker thread — #8983 ‏‏‎‏‏‎ ‎‎ See the release notes for full details on the updates in Prefect 2.9.0!

Hello! Here's a short loom introduction to artifacts. If you've got any feedback or questions, please share in the #prefect-community!

👋 Hi everyone, I tried to sneak this into #show-us-what-you-got but the public response has been pretty overwhelming, so I thought I’d share it more broadly! 🤖 Yesterday we launched a new open-source product called Marvin. (Yes, the same @Marvin you know and love.) Marvin is a library for building AI-powered software, inspired by our own work to deploy AI here in the community, new Prefect features we’re exploring, and working with our customers that are building AI stacks. We wanted a high-level way to focus on using the AI to achieve outcomes that would also let us codify the tips and tricks we’ve been learning along the way to get good results. 🪄 Marvin’s headline feature is AI Functions. These are minimalist functions that have typed inputs, outputs and a docstring, but no source code. Instead, when you call one, the output is predicted by GPT-4 (or GPT-3.5). I know this sounds 🤯 but it actually works astoundingly well. You basically write the

def

of a function you wish you had… and then you call it. 🎁 To get this magic, just add Marvin’s

@ai_fn

decorator. Here’s an example:

from marvin import ai_fn

@ai_fn
def spam_rating(email_body: str) -> int:
    """
    Returns a number between 0 and 100 
    that represents how likely the email is spam
    """
    
    
spam_rating("Hey, just following up about dinner tomorrow.") # 17
spam_rating("Hi, I'm  Nigerian prince and I really need your help.") # 95

⭐️ Obviously, this is cool. But more importantly, it aligns with Prefect’s product mission to eliminate negative engineering and bring engineers as close as possible to their objectives. Give Marvin a star on GitHub and stay tuned — AI functions are just the beginning of what it can do. 💬 Marvin’s community is forming in Discord. Join us there if building with AI is interesting to you! 🚀 Happy engineering!

👋 Join @Jeremiah and @Nate live at noon Eastern today for a surprise livestream as they show off Marvin AI. They'll be covering AI Functions, and I'm sure you'll see some wild

marvin chat

action. Join directly on our YouTube channel or add to your calendar!

Prefect 2.10 has been released! ‏‏‎ ‎ This is a big release with Workers, Projects, Variables, and much more - so read on to to find out what's included. ‏‏‎ ‎‏‏‎ ‎ Prefect deployments often have critical, implicit dependencies on files and build artifacts, such as containers, that are created and stored outside of Prefect. Each of these dependencies is a potential stumbling block when deploying a flow - you must ensure they’re satisfied for your flow to run successfully. In this release, we’re introducing two new beta features, workers and projects, to help you better manage your flow deployment process. ‏‏‎ ‎‏‏‎ ‎ Additionally, we’re releasing variables for centralized management of management and expanding events and automations to include blocks. There are a lot of highlighted features this week — but we’ve also made some significant performance improvements alongside a slew of bug fixes and enhancements! ‏‏‎ ‎ Workers [Beta] Workers are next-generation agents, designed from the ground up to interact with work pools. Each worker manages flow run infrastructure of a specific type and must pull from a work pool with a matching type. ‏‏‎ ‎ Existing work pools are all “agent” typed for backwards compatibility with our agents — but new work pools can be assigned a specific infrastructure type. Specifying a type for a work pool simplifies choosing what kind of infrastructure will be used when creating a flow run. Work pools expose rich configuration of their infrastructure. Every work pool type has a base configuration with sensible defaults such that you can begin executing work with just a single command. The infrastructure configuration is fully customizable from the Prefect UI. ‏‏‎ ‎ For example, you can now customize the entire payload used to run flows on Kubernetes — you are not limited to the fields Prefect exposes in its SDK. We provide templating to inject runtime information and common settings into infrastructure creation payloads. Advanced users can add custom template variables which are then exposed the same as Prefect’s default options in an easy to use UI. ‏‏‎ ‎ If the work pool’s configuration is updated, all workers automatically begin using the new settings — you no longer need to redeploy your agents to change infrastructure settings. For advanced use cases, you can override settings on a per-deployment basis. This release includes Process, Kubernetes, and Docker worker types. Additional worker types will be included in subsequent releases. ‏‏‎ ‎ See the updated work pool, workers, & agents concepts documentation for more information. ‏‏‎ ‎ Projects [Beta] A project is a directory of files that define one or more flows, deployments, Python packages, or any other dependencies that your flow code needs to run. If you’ve been using Prefect, or working on any non-trivial Python project, you probably have an organized structure like this already. ‏‏‎ ‎ Prefect projects are minimally opinionated, so they can work with the structure you already have in place and with the containerization, version control, and build automation tools that you know and love. With projects as directories, you can make relative references between files while retaining portability. We expect most projects to map directly to a git repository. In fact, projects offer a first-class way to clone a git repository so they can be easily shared and synced. ‏‏‎ ‎ Projects also include a lightweight build system that you can use to define the process for deploying flows in that project. That procedure is specified in a new

prefect.yaml

file, in which you can specify steps to build the necessary artifacts for a project’s deployments, push those artifacts, and retrieve them at runtime. ‏‏‎ ‎ Projects are a contract between you and a worker, specifying what you do when you create a deployment, and what the worker will do before it kicks off that deployment. Together, projects and workers bridge your development environment, where your flow code is written, and your execution environment, where your flow code runs. Create your first Prefect project by following this tutorial. See the new project concept doc for more information. ‏‏‎ ‎ Variables Variables enable you to store and reuse non-sensitive bits of data, such as configuration information. Variables are named, mutable string values, much like environment variables. They are scoped to a Prefect Server instance or a single workspace in Prefect Cloud. Variables can be created or modified at any time. ‏‏‎ ‎ While variable values are most commonly loaded during flow runtime, they can be loaded in other contexts, at any time, such that they can be used to pass configuration information to Prefect configuration files, such as project steps. You can access any variable via the Python SDK via the

.get()

method.

from prefect import variables

# from a synchronous context
answer = variables.get('the_answer')
print(answer)
# 42

# from an asynchronous context
answer = await variables.get('the_answer')
print(answer)
# 42

# without a default value
answer = variables.get('not_the_answer')
print(answer)
# None

# with a default value
answer = variables.get('not_the_answer', default='42')
print(answer)
# 42

See the new variables concept doc for more information or the pull request for implementation details. ‏‏‎ ‎‏‎ ‎ Events Continuing the rollout of events as the primary unit of observability in Prefect Cloud, Prefect will now emit events for all block method calls by default. ‏‏‎ ‎‏‏‎ ‎ These events can be viewed in the Event feed, allowing you to analyze the interactions your flows and tasks have with external systems such as storage locations, notification services, and infrastructure. Additionally, you can trigger automations based on these events. For example, you can create an automation that is triggered when a file is uploaded to a storage location. ‏‏‎ ‎ Versioned documentation We’re releasing many new features every week and we know not everyone is on the latest version of Prefect. We’ve added versioning to our documentation website to make it easier to find the docs for the version of Prefect that you’re using. Now, when you visit the Prefect documentation site, you’ll see a version selector at the top of the page. ‏‏‎ ‎ Breaking Changes There’s one breaking change to be aware of in this release: • Unused options for sorting logs have been removed from the API — #7873 ‏‏‎ ‎‏‏‎ ‎ Contributors Thanks so much to the external contributors to this release! • @mianos made their first contribution in #9077! • @dominictarro made their first contribution in #8965! • @joelluijmes • @john-jam ‏‏‎ ‎ Enhancements and Fixes The list of enhancements and fixes this week is long, so check out the 2.10.0 release notes to see them all! We added a quick 2.10.1 release with a fix for projects, but 2.10.0 is the place to read about all the changes.

Prefect 2.10.4 has been released! This update brings you enhancements for project deployments and workers, along with helpful fixes for flow run cancellation and the worker CLI. Let’s dive into the details! ‏‏‎ ‎ 🛠️ More flexible project deployments No more errors when removing keys from a project’s

deployment.yaml

! Thanks to the changes in #9190, Prefect now uses default values for any required keys missing from your project’s configuration. ‏‏‎ ‎ 🕑 Streamlined cancellation for scheduled flow runs In response to challenges cancelling scheduled flow runs, #8414 introduces a new flow orchestration rule. This rule ensures scheduled flow runs that haven’t started and lack an infrastructure PID skip the

CANCELLING

state where they previously got stuck and directly transition to

CANCELLED

. ‏‏‎ ‎ Other Changes: ‏‏‎ ‎ ✨ Enhancements • Add flow run cancellation support for workers - #9198 ‏‏‎ ‎ 🔨 Fixes • Fix

work_queues

and

worker_type

arguments for the

prefect worker start

CLI command — #9154 • Fix overflow in flow run logger UI — #1342 • Fix schema form handling of reference objects — #1332 • Improve flow graph UX by suppressing shortcuts when a metakey is active — #1333 ‏‏‎ ‎ 🧪 Experimental • Emit an event when a worker submits a flow run for execution — #9203 ‏‏‎ ‎ 📚 Documentation • Fix a broken link by removing an obsolete redirect — #9189 • Add polling interval information to worker and agent documentation — #9209 • Update documentation badge styling to improve docs usability — #9207 ‏‏‎ ‏‏‎ ‎ Normally, we’d suggest you head over to the release notes to learn more about Prefect 2.10.4. But after the big 2.10.0 release a week ago, then two quick follow-up releases, we’ve kept 2.10.4 simple — and you’ve already seen everything! 😄

We're excited to share Prefect 2.10.5! G Deploy a Prefect flow via Github Actions With the new Deploy a Prefect flow GitHub Action, you can automate the build process for deployments orchestrated by Prefect Cloud. The action leverages the new Projects system. See the action page for examples and configuration options. 🛠️ Run flows on your favorite cloud provider with Workers Workers, Prefect's next-generation agents 🧪 , have dedicated infrastructure types. This week, we are releasing typed workers for each major cloud provider: aws, gcp, and azure. You will be able to find them in the prefect-aws, prefect-gcp, and prefect-azure collections, respectively. but wait theres more Other fixes and enhancements This is just a sample of the fixes and enhancements from this week's release. Check out the release notes for more information.

Prefect 2.10.6 is out! This release primarily focuses on enhancing the deployment experience and streamlining project setup, but as usual contains various enhancements, fixes, and refinements. Some highlights: 🔀🚀 deploy multiple flows simultaneously using the improved

deployment.yaml

file 🔄🌟 enhanced support for recursive flow calls and concurrent runs of the same flow 📚👨‍💻 a new tutorial on developing custom workers 🤖 Check out the release notes for more info!

We're so excited to share Prefect 2.10.7 🚀 its beautiful This release combines the previously separate flows and deployments UI pages into a single, holistic page that brings together information on flows and deployments, as well as their recent and upcoming runs. You can now see the state of the most recent flow run for each flow and deployment, giving you a snapshot of the status of your workspace. In addition, you can now filter deployments by whether their schedule is active and the work pool to which flow runs are submitted. crash This release also introduces the new

on_crashed

hook for flows, allowing you to add client-side hooks that will be called when your flow crashes. This is useful for cases where you want to execute code without involving the Prefect API, and for custom handling on

CRASHED

terminal states. This callable hook will receive three arguments:

flow

,

flow_run

, and

state

.

from prefect import flow

def crash_hook(flow, flow_run, state):
    print("Don't Panic! But the flow has crashed...")

@flow(on_crashed=[crash_hook])
def my_flow():
    # call `crash_hook` if this flow enters a `CRASHED` state
    pass

Check out the release notes for more information.

Hello all 👋 ✨ Starting today, you’ll begin to see Worker and Block events displayed alongside your Flow Run timelines in Prefect Cloud. Worker events provide context about your execution environment, including environment-specific events such as Kubernetes pod failures and evictions. Any integration activity that uses Blocks will also create an event, such as calls to storage or notification systems. Click into an event to see more information about it or create automations to react to events like this in the future. 🫂 Check it out and let us know what you think! - we’ll be continuing to invest in bringing the maximum amount of actionable context to the UI over the next few months.

Have your voice heard in the design of the new UI landing page/dashboard (if you haven’t yet - lots of great input)! 🚀 https://prefect-community.slack.com/archives/C0192RWGJQH/p1683137878150019

Today, we have a special, off-cycle release: 2.10.8 bogo The release updates flow run orchestration rules to improve orchestration behavior during infrastructure restarts. For example, when a Kubernetes pod kubernetes was interrupted, Prefect would previously abort the run as it attempted to transition from a RUNNING to a RUNNING state. Now, Prefect will allow this transition and your flow run will continue. In particular, the following state transitions have changed: • CANCELLED -> PENDING is not allowed • CANCELLING/RUNNING -> RUNNING is allowed • CANCELLING/RUNNING/PENDING -> SCHEDULED is allowed See #9447 for details. Notably, we have also added a deprecation warning for dropping support for Python 3.7 when it reaches end-of-life on 27 Jun 2023. soon horse See the release notes for more information.

⚠️ Important information if you are still using Prefect Cloud 1 (cloud.prefect.io) AND you are on the Free license tier. ⚠️ As announced earlier this year, tenants on the Free tier of Prefect Cloud 1 will be frozen on May 15, 2023. Starter and Standard tier tenants will be frozen on July 15, 2023. Thousands of Prefect Cloud users have successful migrated to Prefect Cloud 2. Most Prefect 1 flows can be updated to run with Prefect 2 in just a few minutes. To help our remaining users as they transition, we are providing migration support to all users via written guides and help from real Prefect engineers. To get started with migration, signup for a Prefect Cloud 2 account and visit the Migration Station.

We just released Prefect 2.10.10. This release is all about speed ultra fast parrot. In particular, we've sped up CLI performance and imports by as much as 4x on some systems race car! Its enough that you should feel it. This release also adds several enhancements and fixes that make Prefect sleeker and more stable. See the release notes for more information.

Yesterday we released Prefect 2.10.11. This release brings upgrades that streamline your deployment experience! The

prefect deploy

command now offers interactive prompts for deployment names and work pool selections. 👥 We've also introduced a work pool creation wizard 🧙 to simplify the process, guiding you through essentials like basic work pool info, infrastructure type, and configuration. Additional improvements include support for SQLAlchemy 2, and much more! See the release notes for more details.

Yesterday we released 2.10.12 which brings the return of the deployments page! We got a lot of positive feedback about the new flows page that was redesigned to include deployments, but several users pointed out that the it wasn't quite a full replacement for the dedicated deployments page. The deployments page has been re-added to the navigation menu until the new flows page is a worthy substitute. There's lots of other enhancements, fixes and docs updates. Check out the release notes for more!

We just released Prefect 2.10.13, which brings some big improvements to project-based deployments 🚀. You can now run prefect deploy without first initializing a project. If you run prefect deploy without a project initialized, the CLI will generate a default pull step that your worker can use to retrieve your flow code when executing scheduled flow runs. The prefect deploy command will also prompt you with scheduling options, making it even easier to schedule your flows 🕐 . This release also adds two new deployment steps:

pip_install_requirements

and

run_shell_script

. Both are new 'utility' deployment steps that can be used to automate portions of your deployment process 🧰 . For example, use the

pip_install_requirements

step to install Python dependencies before kicking off a flow run:

pull:
    * prefect.projects.steps.git_clone_project:
        id: clone-step
        repository: <https://github.com/org/repo.git>
    * prefect.projects.steps.pip_install_requirements:
        directory: {{ clone-step.directory }}
        requirements_file: requirements.txt
        stream_output: False

Use the

run_shell_script

step to grab your repository's commit hash and use it to tag your Docker image:

build:
    * prefect.projects.steps.run_shell_script:
        id: get-commit-hash
        script: git rev-parse --short HEAD
        stream_output: false
    * prefect.projects.steps.build_docker_image:
        requires: prefect-docker
        image_name: my-image
        image_tag: "{{ get-commit-hash.stdout }}"
        dockerfile: auto

See the release notes for more information.

We're hosting a livestream on June 28th! Add it to your calendar at the link below, and we'll see you there 📺 youtube

We are pleased to announce that yesterday we released Prefect 2.10.14. 🚀 This release simplifies our new style of deployments even further, including the addition of more interactive CLI paths for a smoother deployment experience 🧈 . We also introduce a Prefect Cloud webhook CLI, as well as many more enhancements and bug fixes! 🐛 🪝*Prefect Cloud Webhook CLI* Webhooks on Prefect Cloud allow you to capture events from a wide variety of sources in your data stack, translating them into actionable Prefect events in your workspace. Produce Prefect events from any system that can make an HTTP request and use those events in automations or to trigger event-driven deployments. Even if you have minimal control over the systems you're integrating with, Prefect Cloud webhooks give you full programmable control over how you transform incoming HTTP requests into Prefect events with Jinja2 templating. We even have a built-in preset for CloudEvents. Webhooks are currently available via the API and `prefect` CLI. You can create your first Cloud webhook via the CLI like so:

prefect cloud webhook create your-webhook-name \
    --description "Receives webhooks from your system" \
    --template '{ "event": "your.event.name", "resource": { "prefect.resource.id": "your.resource.id" } }'

For more information, see the release notes! 🎉

👋 Hey there! If you're stuck with a Prefect-specific problem, or just want some data engineering help in general: we have a Product Advocate team that can help you "rubber duck" your problems and offer suggestions/technical advice. If you'd like to grab some time, you can use this link!

Cross-posting: a great opportunity to follow along with the new

prefect init

!