Great question — I’m happy to break this down. I want to make sure I give you precise, source-linked notes for each version. Could you share the link where you’re seeing 0.3.1–0.3.5 (PyPI or GitHub)? My tooling returned conflicting data about these exact tags, so I’d like to confirm the authoritative source before I summarize per-version changes. In the meantime, here are the best places to verify and compare versions yourself: - PyPI release history: prefect-shell on PyPI - Source in the Prefect monorepo: prefect-shell source in Prefect repo - Prefect releases/tags (you can search for tags like

prefect-shell-0.3.x

): Prefect tags If you can confirm the page you’re looking at, I’ll post a concise, per-version summary (0.3.1 → 0.3.5) with direct links to the exact release notes/changelogs in my next reply.

@Marvin show me the github release notes, there are no release notes in PyPI

Got it — here are the GitHub release/tag pages for prefect-shell 0.3.1 through 0.3.5. Prefect publishes integration versions as git tags in the main Prefect repo; some tags have only a brief tag message (no separate GitHub “Release” object). You can also use the compare links to see the exact diffs. - 0.3.1 - Tag: prefect-shell-0.3.1 - Compare: 0.3.0...0.3.1 - 0.3.2 - Tag: prefect-shell-0.3.2 - Compare: 0.3.1...0.3.2 - Docs release notes: prefect-shell release notes (scroll to 0.3.2) - 0.3.3 - Tag: prefect-shell-0.3.3 - Compare: 0.3.2...0.3.3 - Docs release notes: prefect-shell release notes (scroll to 0.3.3) - 0.3.4 - Tag: prefect-shell-0.3.4 - Compare: 0.3.3...0.3.4 - 0.3.5 - Tag: prefect-shell-0.3.5 - Compare: 0.3.4...0.3.5 Notes: - For some versions (commonly 0.3.1, 0.3.4, 0.3.5), the tag page serves as the “release notes” and may only include a short tag message. - The most complete, curated notes (when available) are published in the docs: prefect-shell release notes. If you’d like, I can fetch and paste the current tag messages from those pages directly into this thread.

Short answer: yes — a couple of the 0.3.x changes can affect what you see in your terminal, mostly around how quickly output appears and how it’s routed. What may impact console output - 0.3.4: Improved incremental streaming for synchronous runs. You should see stdout/stderr appear more “live” (less buffering) when

stream_output=True

. - Tag: prefect-shell-0.3.4 - 0.3.5: Fix to preserve streamed logs in the Prefect API. This mainly affects what shows up in the UI/API; terminal output should be unchanged unless you rely on API round‑tripped logs for local display. - Tag: prefect-shell-0.3.5 - 0.3.2: Internal migration to async_dispatch changed execution paths for sync/async under the hood. The default behavior still streams when

stream_output=True

, but timing/ordering could feel slightly different in mixed async/sync contexts. - Tag: prefect-shell-0.3.2 Unlikely to affect console output - 0.3.3: Dependency floor bump only. - Tag: prefect-shell-0.3.3 - 0.3.1: No known change specifically targeting shell log streaming. - Tag: prefect-shell-0.3.1 Quick checks - Ensure

stream_output=True

(default remains True): - Confirmed by the current API:

ShellOperation(..., stream_output: bool = True, ...)

- Verify Prefect logging level/handlers. By default, Prefect prints logs to the console when you run flows locally. - If you upgraded and now see delayed output, 0.3.4 should improve real‑time streaming for sync commands. Minimal example to confirm streaming locally

from prefect import flow
from prefect_shell import shell_run_command

@flow
def demo():
    shell_run_command("for i in 1 2 3; do echo $i; sleep 1; done", stream_output=True)

if __name__ == "__main__":
    demo()

If you’re seeing unexpected behavior, let me know: - Your prefect-shell version - Where you’re running (local vs worker, and which worker) - Whether the issue is “no logs at all” or “logs appear only at the end” or “logs missing in the UI but present locally”

@Marvin What we noticed is that in prefect-shell 0.3.5 with prefect 3.4.11 we stopped seeing really long log outputs on the prefect UI console. We run a command with ShellOperation(commands=[command]).run()

Thanks for the details — that helps narrow it down. What’s likely happening - prefect-shell 0.3.4/0.3.5 changed how sync output is streamed (threaded, line-by-line) and 0.3.5 specifically added a fix to preserve streamed logs in the API. Those changes were developed/tested against newer Prefect 3.x logging behavior. - On Prefect 3.4.11, the logging pipeline and context propagation are older. With 0.3.5’s new threaded streaming, long/high‑volume output may not make it to the API/UI reliably on 3.4.11 — you’ll see less (or none) of those “really long” logs in the UI even though the process ran. Why it shows up with “really long” output - When output is emitted as a large burst (or as many rapid, small log records), older Prefect versions can drop/clip logs sent to the API (payload size, rate limiting, and missing context propagation). Newer Prefect versions added better max-size handling and context propagation — which 0.3.5 assumes. Recommended fixes (in order) 1) Prefer: upgrade Prefect to a newer 3.x (ideally latest 3.x, at least 3.6.x) - This aligns your Prefect logging pipeline with the streaming changes in prefect-shell 0.3.5 so long outputs reach the UI as many small records rather than a single huge one. - Tag refs for context: - 0.3.4 (streaming improvements): prefect-shell-0.3.4 - 0.3.5 (preserve streamed logs in API): prefect-shell-0.3.5 - Integration release notes (curated): prefect-shell release notes 2) If you must stay on Prefect 3.4.11, try one of these workarounds: - Downgrade prefect-shell to a 0.3.x version that works better with your stack (0.3.2 or 0.3.3 are safer bets when paired with older Prefect). - Temporarily disable streaming and log in chunks yourself. This avoids the threaded streaming path that can drop messages on 3.4.11:

from prefect import flow, get_run_logger
  from prefect_shell import ShellOperation

  def log_chunks(logger, text, size=8000):
      for i in range(0, len(text), size):
          <http://logger.info|logger.info>(text[i:i+size])

  @flow
  def run_and_log():
      logger = get_run_logger()
      op = ShellOperation(commands=["your-long-command"], stream_output=False)
      lines = op.run()  # returns list[str]
      output = "\n".join(lines)
      log_chunks(logger, output)

  if __name__ == "__main__":
      run_and_log()

- This ensures the UI receives many smaller log records, which are less likely to be dropped/truncated. Quick confirmations - Keep

stream_output=True

for normal usage (it’s the default), unless you’re applying the “log chunks” workaround above. - Verify logs are being sent to the API in your runtime environment (Cloud vs self-hosted), and whether you can see the full logs locally (stdout) but not in the UI — that’s a strong signal it’s the API/logging pipeline, not the shell process itself. If you can share: - Your exact versions (prefect, prefect-shell), worker type (process/docker/k8s), and whether you’re on Cloud or a self-hosted server - Whether you see the full output locally but not in the UI …I can recommend the safest pin (upgrade path vs. specific downgrade) for your setup.

Here’s a practical view of what can break or require changes when moving from Prefect 3.4.x to 3.6.x. I’ve grouped the items by impact, with links to the official notes where available. Highest-impact breaking changes - Python 3.9 support removed (3.5.0) - You must run Python 3.10+. - Link: 3.5.0 release notes - System Blocks and legacy Runner APIs removed (3.5.0) - Removed system blocks:

prefect.blocks.system.{JSON,String,DateTime}

→ migrate to Variables. - Removed Runner utilities (e.g.,

submit_to_runner

,

wait_for_submitted_runs

) and deprecated server helpers. - Some deployment control helpers removed/renamed (pause/resume via API/CLI instead). - Link: PR #19307, 3.5.0 notes - CLI framework change under the hood: Typer → Cyclopts (3.6.20+) - End-user CLI commands/flags remain the same. - If you’ve built custom CLI plugins that import/use Typer internals from Prefect, you’ll need to migrate to Cyclopts. - Links: PR #20821, PR #20838 Other behavior/compatibility changes to be aware of - Legacy settings/internal imports fully removed (3.6.0) - Code importing private settings modules or legacy settings paths will break. Use documented

PREFECT_*

env vars and public settings APIs. - Link: PR #19353 - Background task system updated (3.6.0) - Server background tasks switched to a more robust mechanism (with optional Redis backing). Mostly transparent, but custom server extensions relying on FastAPI BackgroundTasks may require adjustment. - Link: PR #19377 - Logging pipeline improvements in 3.6.x (affects very long logs) - Newer 3.6.x releases improved max-log-size handling and API propagation. If you rely on extremely long single log records, expect truncation at cloud limits (25k chars) and consider chunking. This is typically beneficial but can change observed behavior compared to 3.4.x. - Notes: see 3.6 release notes stream 3.6.x notes - Integration compatibility floors raised - Many integration packages (e.g., prefect-shell >=0.3.3) started requiring Prefect >=3.6.17. If you upgrade integrations while staying on 3.4.x, you’ll hit resolver conflicts or subtle behavior differences. - Example PR: PR #20659 What typically doesn’t change for most 3.4 → 3.6 upgrades - Core Flow/Task APIs remain stable. - Deployments, workers, and pools keep the same user-facing commands and configuration shapes (beyond internal CLI framework change). - Prefect Cloud account/org/workspace behavior is unaffected. Suggested upgrade path and checks - Environment - Ensure Python 3.10+ before upgrading Prefect. - If you’ve pinned integrations (prefect-aws, prefect-kubernetes, prefect-shell, etc.), verify their Prefect version requirements first. - Codebase scans - Remove system blocks usage: - Replace

prefect.blocks.system.*

with

prefect.variables.Variable

. - Remove runner APIs: - Replace any imports from

prefect.runner

or references like

submit_to_runner

. - Check for ad-hoc settings imports: - Replace private settings imports with env vars (

PREFECT_*

) or public config accessors. - If you have custom CLI plugins: - Replace Typer usage that depends on Prefect’s CLI internals with Cyclopts-compatible code (or shell out to

prefect

). - Logging for very long outputs - Prefer line-by-line logging (or chunk large strings to <25k chars) for Cloud. - If you use prefect-shell for long-running commands, use recent versions (0.3.4/0.3.5) with Prefect 3.6.x to get reliable incremental streaming to the UI.

Useful links - 3.5.0: release notes - 3.6.x: release notes stream - PRs highlighting breaking areas: - Remove system blocks &amp; runner APIs - Remove legacy settings imports - Drop Python 3.9 - Remove Typer from CLI - Promote Cyclopts If you share which features you’re using (system blocks, any runner utilities, custom CLI bits), I can give you a targeted diff and migration snippets.

Short answer: - Row-by-row streaming has been the intended behavior since ShellOperation was first introduced. - In the 0.3.x line there was a brief regression: 0.3.2 changed the sync code path to use a fully blocking subprocess wait, which buffered output and could appear as one large chunk. - 0.3.4 restored true incremental streaming for the sync path; 0.3.5 fixed an issue where those streamed lines sometimes didn’t make it to the API/UI. Pointers you can use to verify: - 0.3.2 (sync path refactor that introduced buffering for sync runs): prefect-shell-0.3.2 - 0.3.4 (incremental streaming for sync

wait_for_completion

): prefect-shell-0.3.4 - 0.3.5 (preserve streamed logs in Prefect API): prefect-shell-0.3.5 - Curated notes when available: prefect-shell release notes So, if you were on prefect-shell 0.3.1 (row-by-row) and then noticed grouped output, that change in behavior lines up with 0.3.2. If you need row-by-row on the sync path, 0.3.4+ restores it (and 0.3.5 ensures those streamed lines reach the UI).

Totally fair expectation — with prefect-shell 0.3.5 you should get row‑by‑row logs. The reason you’re not seeing those long streams in the UI with Prefect 3.4.11 boils down to how 0.3.5 streams logs vs. how 3.4.11 sends/associates logs to the API/UI. What changed in 0.3.4/0.3.5 - 0.3.4 switched sync runs to truly stream incrementally (stdout/stderr line‑by‑line) instead of buffering and emitting at the end. - Tag: prefect-shell-0.3.4 - 0.3.5 fixed a bug so those streamed lines are sent to the Prefect API with the right logging context from background threads. - Tag: prefect-shell-0.3.5 Why that can fail on Prefect 3.4.11 - Thread context vs. 3.4.11 logging: 0.3.5 emits logs from child threads, relying on the Prefect logging context to associate each record to the active flow run. On 3.4.11, context propagation from background threads isn’t as robust; some log records lose run context and never appear in the UI even though the process ran and you may see local stdout. - Volume/rate behavior: Row‑by‑row streaming produces many small log records quickly. Older 3.4.11 API/logging pipelines can drop or de‑prioritize high‑throughput log bursts, which disproportionately affects long outputs. - Truncation/visibility differences: With 0.3.1 you often got one or a few large records at the end; with 0.3.5 you get many small records. The older pipeline can surface less content (or only tails) when lots of small records arrive. What you can do - Best fix: upgrade Prefect to a newer 3.6.x (or latest 3.x). That aligns the logging pipeline with the 0.3.5 streaming approach so row‑by‑row lines reliably reach the UI. - 3.6.x release notes: 3.6.x notes - If you must stay on 3.4.11: - Pin prefect-shell back to 0.3.1 (the behavior you liked) or 0.3.4 (incremental streaming) and test which works better with your setup. - Or bypass threaded streaming and log in controlled chunks:

from prefect import flow, get_run_logger
    from prefect_shell import ShellOperation

    def log_chunks(logger, text, size=8000):
        for i in range(0, len(text), size):
            <http://logger.info|logger.info>(text[i:i+size])

    @flow
    def run_and_log():
        logger = get_run_logger()
        op = ShellOperation(commands=["your-long-command"], stream_output=False)  # capture but don't stream
        lines = op.run()  # list[str]
        output = "\n".join(lines)
        log_chunks(logger, output)

    if __name__ == "__main__":
        run_and_log()

- Quick checks that help diagnose: - Ensure

stream_output=True

if you want native streaming. - Confirm logs are being sent to the API in your runtime (look for “Sending logs to API” on worker start or ensure

PREFECT_LOGGING_TO_API=true

). - Verify log level is INFO (e.g.,

PREFECT_LOGGING_LEVEL=INFO

). - Check worker stdout vs. UI: if you see the full stream locally but not in the UI, it’s the API/logging pipeline issue above. If you can share your worker type (process/docker/k8s), whether you’re on Cloud vs. self‑hosted, and whether the full output appears locally but not in the UI, I can suggest the safest pin/upgrade path for your exact setup.