wave hello from Chicago At Saturn Cloud

Question

wave hello from Chicago At Saturn Cloud we offer a managed solution for data scientists and data engineers to easily provision and manage Dask clusters so they can speed up their Python workflows We ve recently added an integration with Prefect Cloud and I m really excited to share it here tada The workflow we ve designed is like this Provision a `KubernetesAgent` with a few clicks in the Saturn UI Author your flow in a Saturn managed instance of Jupyter Lab Register the flow with Prefect Cloud A library called `prefect saturn` adds an environment and storage to your flow that says hey run this on one of my Saturn Dask clusters Whenever your flow runs it creates a Dask cluster for itself programmatically All your dependencies files and necessary credentials will be available on all of the Dask workers and all logs and task statuses get sent back to Prefect Cloud More details about the architecture and links to sample code are available at I want to thank the `prefect` team for the care and attention they ve given to keeping `prefect` modular We were able to write a client library like this and another thin wrapper around `KubernetesAgent` without needing to manage a lot of code because of the clear separation of concerns in the `prefect` library The maintainers have also been very responsive to my questions in this Slack and to my proposals in issues and pull requests heart