Hui Zheng
11/04/2020, 7:38 PMhttps://github.com/PrefectHQ/prefect/issues/3614please see this link or the thread for the details
dbt run
as shell sub-process. I then use another task, called parse_dbt_result
, to take the output of the dbt task and handle both success and failure results, such as knowing which models in the dbt-run process failed.
When some models fails in a dbt run
, those model errors are shown in the stdout, and the shell subprocess returns non-zero exit code (1). The ShellTask raises a FAIL signal without any stdout result. As a result, the parse_dbt_result
task is not able to interpret the log of failed run and not know which model fails, hereby not be able to handle the failure properly.
Solution
please provide a flag that allows ShellTask to return stdout(or/and stderror) result when the shell process fails and FAIL signal is raised.