Hi there! Another question. We recently did an upgrade to our prefect instance, and post upgrade some logs are no longer getting pulled in from the running task. We use a very lightweight wrapper around the DbtShellTask: https://docs.prefect.io/api/latest/tasks/dbt.html#dbtshelltask
And we have the log_stdout=True variable set which according this this should let the stdout show up:
https://docs.prefect.io/core/idioms/logging.html#logging-stdout
I’ve ssh’d into the running task and verified that its emitting the stdout we are expecting, is there a step I should be looking at somewhere? Thank you!
a
Anna Geller
04/29/2022, 5:44 PM
Good question! By default, dbt logs are not emitted directly when you run this task. You would need to capture those via a separate task.
Copy code
@task(trigger=all_finished)
def print_dbt_output(output):
logger = prefect.context.get("logger")
for line in output:
<http://logger.info|logger.info>(line)
for a full example, check this blog post and LMK if you still have some questions about that
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.