We do have two problems with Prefect flow when using Prefect-Databricks:
• the job_sumit_and_wait flows uses subflows that ends up polluting the UI (who cares about how many times you polled to get the result from Databricks?)
• the information to find the Databricks job are not located in our parent job that really submit the job on Databricks, but we need to navigate to child flos