👋 hello from Chicago! At Saturn Cloud (https://www.saturncloud.io/s/), we offer a managed solution for data scientists and data engineers to easily provision and manage Dask clusters, so they can speed up their Python workflows. We've recently added an integration with Prefect Cloud, and I'm really excited to share it here! 🎉 The workflow we've designed is like this: • Provision a

KubernetesAgent

with a few clicks in the Saturn UI • Author your flow in a Saturn-managed instance of Jupyter Lab • Register the flow with Prefect Cloud. A library called

prefect-saturn

(https://github.com/saturncloud/prefect-saturn) adds an environment and storage to your flow that says "hey, run this on one of my Saturn Dask clusters" • Whenever your flow runs, it creates a Dask cluster for itself programmatically. All your dependencies, files, and necessary credentials will be available on all of the Dask workers, and all logs and task statuses get sent back to Prefect Cloud. More details about the architecture and links to sample code are available at https://www.saturncloud.io/docs/connecting/tools/prefect-cloud/. I want to thank the

prefect

team for the care and attention they've given to keeping

prefect

modular. We were able to write a client library like this (and another thin wrapper around

KubernetesAgent

) without needing to manage a lot of code, because of the clear separation of concerns in the

prefect

library. The maintainers have also been very responsive to my questions in this Slack and to my proposals in issues and pull requests. ❤️