Just a note to say, props to @Kevin Kho for giving me some pro-tips on Prefect setup in combination with databricks using delta lake. I have orchestrated a bunch of our jobs through our data layers now and setup a strategy for our developer loop using a combination of a local cluster for unit tests and databricks-connect for our on cluster runs, either via local machines or deployed to AKS. Not production ready yet, but huge strides. Couldnt have done it without your help ❤️

Thanks! Anytime!