Just did a presentation in my company about prefect. All the slide (as Jupyter notebook) and code is available on my github. Please enjoy and feel free to use it if you have a presentation to do. https://github.com/davzucky/prefect_presentation

@Lucas Hosoya, this might interest your team?

@Kevin Kho Hey! Yup, it does. But we also want some topics other than Dask environments, if possible. Maybe some kubernetes with databricks/spark/hdinsight/glue (run outside of the Agent, so we can put the work on some clusters other than the Agent) But this presentation is great!

@Lucas Hosoya what are you looking for? Presentation or usage using kubernetes/spark?

@davzucky Hey! Kubernetes/spark would be great to see! Also looking forward to see the usage with a Cloud environment, if possible. Great presentation though, thanks for sharing! PS: I'm actually doing this case as well, but I'd love to see it from someone else!

We are using all our setup on kubernetes and will run soon spark from prefectv in kubernetes. We need some updated RBAC to get the kuberbetesjobrun task working. Is this what you are looking for?

True! Right now we're about to run kubernetes with databricks instead, due to the spark environment configuration. So everything would run through databricks-connect API, and use the kubernetes agents to orchestrate ETLs/pipelines