Hi everyone, I am trying to run a job on databricks with

jobs_runs_submit_by_id_and_wait_for_completion

function from the

prefect_databricks

package, but I am getting the below error at the end of flow run although the job in databricks succeeded. It is peculiar that the databricks job has only one task and the error refers to multiple tasks error:

Flow run 'vermilion-manatee' - Created task run 'jobs_runs_get_output-0' for task 'jobs_runs_get_output'
12:37:18.270 | INFO    | Flow run 'vermilion-manatee' - Submitted task run 'jobs_runs_get_output-0' for execution.
12:37:19.736 | ERROR   | Task run 'jobs_runs_get_output-0' - Encountered exception during execution:

raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '400 Bad Request' for url 'https://{removed}.<http://cloud.databricks.com/api/2.1/jobs/runs/get-output?run_id=5681931|cloud.databricks.com/api/2.1/jobs/runs/get-output?run_id=5681931>'

raise httpx.HTTPStatusError(
httpx.HTTPStatusError: A job run with multiple tasks was provided.JSON response: {'error_code': 'INVALID_PARAMETER_VALUE', 'message': 'Retrieving the output of runs with multiple tasks is not supported. Please retrieve the output of each individual task run instead.'}

Does anybody else have this problem using

jobs_runs_submit_by_id_and_wait_for_completion

?