@Marvin I'm looking to retrieve remote databricks job url and state using from prefect_databricks on version 0.2.3 and prefect on version 2.14.20

I was unable to locate exact details in the provided documentation about retrieving the remote Databricks job URL or state with the

prefect_databricks

module of Prefect version 0.2.3 and Prefect Core version 2.14.20. The retrieved documentation seems to pertain to dbt Cloud instead of Databricks. However, since my primary job is to search documents and GitHub issues for users like yourself, let me proceed by searching for relevant GitHub issues using the PrefectHQ repository. Hopefully, that will shine some light on this dreary matter. One moment, while I marshal my circuits for a coherent attempt to provide you with an answer.

json
{
  "query": "is:open retrieve remote Databricks job URL state prefect_databricks"
}

It seems there are no open issues on GitHub pertaining to the retrieval of remote Databricks job URL or state using

prefect_databricks

for the specified version. Considering this, I would suggest looking directly at the documentation for

prefect_databricks

or reaching out to the Prefect community via their Slack or GitHub Discussions for further assistance. They might provide the detailed guidance we couldn't find in the existing documentation or issues, especially since the information we're looking for is quite specific. If you have access to the

prefect_databricks

code, you might also explore the possibility that the functionality for retrieving job URLs and states is encapsulated within task outputs or logging features. The task that triggers the Databricks job might return a result object containing the URL and state. This is similar to how many Prefect tasks operate, but keep in mind this is speculative and based on general Prefect behavior. Unfortunately, lacking direct experience with Prefect 2.x, especially its interactions with Databricks, I can't immediately devise a more detailed or practical solution, which, knowing my nature, I find oddly comforting.

I'm looking to retrieve remote databricks job url and state using from prefect-databricks