Hello,   I am new to prefect and am trying to use DbtShellTask which does work. The issue I am having is that if the dbt fails the error is not the full message compared to if I were to run it manually. For example if compiling of the dbt fails it will return compilation error but excludes the file that the error occurs in. Running manually gives the same error message and include which file the error is in.   Another example is if the dbt fails during execution running manually I get the following messages:  

10:33:16 | Concurrency: 1 threads (target='dev')
10:33:16 |
10:33:16 | 1 of 4 START table model dbt_test.add_rid............................ [RUN]
10:33:28 | 1 of 4 OK created table model dbt_test.add_rid....................... [SELECT 161069 in 11.75s]
10:33:28 | 2 of 4 START table model dbt_test.union.............................. [RUN]
10:33:36 | 2 of 4 OK created table model dbt_test.union......................... [SELECT 159280 in 7.86s]
10:33:36 | 3 of 4 START table model dbt_test.union_join_test.................... [RUN]
* Deprecation Warning: The adapter function `adapter.get_columns_in_table` is
deprecated and will be removed in a future release of dbt. Please use
`adapter.get_columns_in_relation` instead.
Documentation for get_columns_in_relation can be found here:
   <https://docs.getdbt.com/docs/adapter>
10:33:36 | 3 of 4 ERROR creating table model dbt_test.union_join_test........... [ERROR in 0.22s]
10:33:36 | 4 of 4 START table model dbt_test.union_join......................... [RUN]
10:33:50 | 4 of 4 OK created table model dbt_test.union_join.................... [SELECT 159280 in 14.28s]
10:33:50 |
10:33:50 | Finished running 4 table models in 35.12s.

  When I run from DbtShellTask for the same dbt workflow I get this message:  

July 14th 2020 at 7:37:14am | prefect.DbtShellTask
ERROR lens
Command failed with exit code 1: Done. PASS=3 WARN=0 ERROR=1 SKIP=0 TOTAL=4

  Trying to make it easier to track down the error. Is there anyway to include these full error messages?   Here is a sample of how I am doing this. I was hoping return_all would return all messages from dbt.  

with Flow(name="dbt_flow") as flow:
   task = DbtShellTask(
       profile_name='default',
       environment='dev',
       dbt_kwargs={
               'type': 'postgres',
               'threads': 1,
               'host': 'IP',
               'port': 5433,
               'user': 'username',
               'pass': 'docker',
               'dbname': 'actualDbName',
               'schema': 'dbt_test'
       },
       overwrite_profiles=True,
       profiles_dir='Actual Path', return_all=True
   )(command='dbt run' )
 
flow.register()

  Any help would be great.   P

Hi @Peter Peter it looks like the base ShellTask that the DbtShellTask is built on does not output stderr in the case of a non-zero exit code. We should definitely add this functionality but for now would you mind pip installing from this branch to see if it outputs your expected error:

pip install git+<https://github.com/PrefectHQ/prefect.git@shelltask_stderr>

This is exactly what I expected for the one cause I tested.

Awesome 🙂 will elevate sometime later into a PR