I have a (I think simple) question around exposing more complete output to the prefect logger – I'm deliberately crashing a task that uses a

DbtShellTask

to execute a dbt project under the hood, and am not supplying the necessary ENV vars that

dbt

needs to run. If I have prefect run my flow that calls this shell task, the only failure message I see is the very last line of what dbt spits out, i.e.

[2021-01-27 16:34:13+0000] ERROR - prefect.run_dbt | Command failed with exit code 2
[2021-01-27 16:34:13+0000] ERROR - prefect.run_dbt |   Could not run dbt

this is obviously not terribly informative, and I'd like for prefect to forward to me the entirety of the error message that dbt generates, which is:

Encountered an error while reading profiles:
  ERROR Runtime Error
  Compilation Error
    Could not render {{ env_var('DB_USERNAME') }}: Env var required but not provided: 'DB_USERNAME'
Defined profiles:
 - default

For more information on configuring profiles, please consult the dbt docs:

<https://docs.getdbt.com/docs/configure-your-profile>

Encountered an error:
Runtime Error
  Could not run dbt

what kind of additional steps need to be taken in order for me to have prefect forward all the lines of output rather than just the final one? it's not clear to me where or how I would need to adjust the default prefect logger to set this up

Hey @Sean Talia I think it’s worth calling out (and not well documented actually!) that the Dbt task inherits from the

ShellTask

which has options for logging more information from stderr 🙂 https://docs.prefect.io/api/latest/tasks/shell.html#shelltask

ah yes i knew I had seen this at some point in the distant past (i.e. 2 weeks ago) – awesome, thank you!