Log truncation with DbtCoreOperation after upgrading from Prefect 3.4.9 to 3.6.x We’re seeing log messages truncated in the Prefect UI and API when running large dbt builds via

DbtCoreOperation

. With 765+ nodes, output from node 5 onward is dropped with a

[truncated]

marker. This was not happening on Prefect 3.4.9 -- logs were complete in the UI before the upgrade. Setup: -

prefect==3.6.x

-

prefect-dbt==0.7.4

-

PREFECT_LOGGING_TO_API_MAX_LOG_SIZE=10000000

What we know: -

DbtCoreOperation

extends

ShellOperation

and pipes raw dbt stdout to Prefect’s log API - Truncation is confirmed at the API level --

prefect flow-run logs

also shows truncated output, so it’s not a UI rendering issue - Logs are complete in the GitHub Actions console, confirming the subprocess output itself is fine - Raising

PREFECT_LOGGING_TO_API_MAX_LOG_SIZE

has not resolved the issue Question: Did

ShellOperation

log handling change between 3.4.x and 3.6.x -- specifically how stdout lines are batched and sent to the log API? Is this a known regression, and if so is there a recommended fix short of switching to

dbtRunner

for all commands? Any guidance appreciated.

Hey! are you self hosting your server or using Prefect Cloud?

Prefect Cloud

I believe

prefect==3.6.4

included this PR https://github.com/PrefectHQ/prefect/pull/19496. Prefect Cloud has always had max single log size it accepts. Prior to this change these logs were sent to Cloud and silently rejected for being too big. Now they're just truncated but are actually accepted. If theres some logging from an integration that you'd like to see chunked, I'd definitely encourage you to open a feature request!

Prior to upgrading, I was able to see the full set of logs in the Prefect Cloud UI - so if some

dbt build

command had 765 nodes, I could see the message from each node. After upgrading, I was getting the truncation message from using DbtCoreOperation. I’m experimenting with setting stream_output=false (although this is a pain for local dev) and handling the dbtRunner myself, although that looks to cause some memory issues running in process. My hope is that there is a simple way to run a dbt command like

dbt build

and have all the nodes show up in the Prefect UI as the command is running. Is there something I am missing here?

What specific version of

prefect

did you upgrade to? It looks like there was a regression in

3.6.14

that was fixed in

3.6.22

where output could be consolidated into one single log instead of many: https://github.com/PrefectHQ/prefect/pull/20739

3.4.9 -> 3.6.21. Let me try 3.6.22

🤔 hm that is definitely not expected. Prefect Cloud has always had had a cap on the max size of a single log payload it accepts, so it does sound like there is a regression that has not been addressed in one of the client libraries to send things as one log vs many logs. Could you open an issue here with what you're seeing? Specific versions and examples of flow runs with truncation/not truncation (and an MRE if you have) would be very much appreciated

Thanks @Jake Kaplan, added here: https://github.com/PrefectHQ/prefect/issues/21409

This is great thank you! Could you actually try one more thing for me and update

prefect-shell

and

prefect-dbt

to the latest versions?

well that looked to fix my MRE, will check shortly on my project