@Kehinde Adewusi Has the question: When using Prefect 1 databricks integration, is it possible return the logs from the databricks jobs so that they show up in the Prefect logs in the Prefect UI? Can flow run logs be sent from Prefect to elastic search?

To get additional logs from python libraries in 1.0 you can add them to the extra loggers. Although I'm not sure if Databricks has a python library connection. To get flow run logs from Prefect to elastic search you could query the GraphQL or while running the flow, output the logs with stdout directly to the elastic search. Most commonly I've seen an additional flow that runs periodically that queries the GraphQL api for flow run logs and then puts them in elastic search.

Thanks @Kalise Richmond, I previously checked the

extra loggers

. perhaps Databricks is just lacking in the extra loggers support? Thanks for pointing out the GraphQL API, I think it's yet another reason to migrate to simpler Prefect 2 REST APIs.