https://prefect.io logo
Join Slack
Powered by
<@ULVA73B9P>: From what I see, there's no notion o...
# marvin-ai
c
@Marvin: From what I see, there's no notion of a Flow/Task State that is along the lines of "Completed with Errors" or "Completed with Warnings", correct?
m
✅ thought for 75.4 seconds
Correct — in Prefect 3.x there is not a built-in state type like 
Completed with Errors
or 
Completed with Warnings
. Built-in state types are things like: - 
COMPLETED
- 
FAILED
- 
RUNNING
- 
PENDING
- 
SCHEDULED
- 
CANCELLED
- 
CRASHED
- 
PAUSED
- 
CANCELLING
So if you're asking whether Prefect has a first-class orchestration state for “success, but imperfect” — no, not as a distinct built-in state type. What people usually do instead is one of these: - return a 
COMPLETED
state with a custom 
message
- use a custom 
name
like 
CompletedWithWarnings
- include warning/error details in the result or 
state.data
- explicitly fail if “partial success” should be treated as operational failure Example pattern:
Copy code
from prefect.states import Completed

return Completed(
    name="CompletedWithWarnings",
    message="Loaded 95/100 records; 5 were skipped",
    data={"skipped": 5, "errors": [...]}
)
That will still behave as a completed run from Prefect’s perspective, but gives you richer context in the UI / API. So the short answer is: - No built-in 
Completed with Errors/Warnings
state type - Yes, you can represent that semantic with a custom completed state name/message/data If you want, I can also show the best practice pattern for: 1. task-level partial failures, or 2. flow-level “green but needs attention” reporting.
c
@Marvin: Please show me the best practice pattern for: 1. task-level partial failures, or 2. flow-level “green but needs attention” reporting.
m
✅ thought for 18.9 seconds
Yep — here are the patterns I’d recommend in Prefect 3.x. Rule of thumb: - Use 
FAILED
when downstream orchestration should treat the run as unsuccessful - Use 
COMPLETED
when the run should stay green, even if there were recoverable issues - Put the nuance in the result, state message, logs, or a custom state name --- 1. Best practice for task-level partial failures This is the cleanest pattern when a task processes many items and some fail, but you do not want the whole task to fail. Have the task: - catch per-item exceptions - collect successes and failures - return a structured result - optionally return a custom 
Completed(...)
state if you want UI messaging Example:
Copy code
from dataclasses import dataclass
from prefect import task, flow
from prefect.states import Completed

@dataclass
class BatchResult:
    succeeded: list
    failed: list

@task
def process_batch(items):
    succeeded = []
    failed = []

    for item in items:
        try:
            result = item * 2  # replace with real work
            succeeded.append({"item": item, "result": result})
        except Exception as exc:
            failed.append({"item": item, "error": str(exc)})

    return Completed(
        name="CompletedWithWarnings" if failed else "Completed",
        message=f"{len(succeeded)} succeeded, {len(failed)} failed",
        data=BatchResult(succeeded=succeeded, failed=failed),
    )

@flow
def my_flow():
    state = process_batch([1, 2, 3], return_state=True)

    result = state.result()
    print(result.succeeded)
    print(result.failed)
Why this is good: - the task remains green if partial failure is acceptable - the failure details are explicit and structured - the flow can inspect and decide what to do next Even simpler option: just return a normal Python object instead of a custom state:
Copy code
from prefect import task

@task
def process_batch(items):
    succeeded = []
    failed = []

    for item in items:
        try:
            succeeded.append(item * 2)
        except Exception as exc:
            failed.append({"item": item, "error": str(exc)})

    return {
        "succeeded": succeeded,
        "failed": failed,
        "warning": f"{len(failed)} items failed" if failed else None,
    }
This is often the best default unless you specifically want the state/message shown in the UI. --- Recommended guideline for task-level partial failures Use a normal return value when: - consumers of the task need structured output - partial failure is part of expected business logic Use a custom 
Completed(...)
state when: - you want the Prefect UI state/message to clearly show warnings - you want operators to notice “green, but imperfect” runs quickly Use 
FAILED
when: - any partial failure should stop downstream work - data quality requires all-or-nothing behavior --- 2. Best practice for flow-level “green but needs attention” reporting This is usually best handled by: - aggregating warning/error counts from tasks - logging clearly - finishing the flow as 
COMPLETED
- optionally returning a custom completed state for a UI-visible summary Example: ``` from prefect import flow, task from prefect.states import Completed from prefect.logging import get_run_logger @task def extract(): return { "records_read": 100, "warnings": ["2 malformed rows skipped"], "errors": [] } @task def transform(data): return { "records_written": 98, "warnings": ["1 duplicate dropped"], "errors": [] } @flow def etl_flow(): logger = get_run_logger() extract_result = extract() transform_result = transform(extract_result) warnings = [] errors = [] warnings.extend(extract_result.get("warnings", [])) warnings.extend(transform_result.get("warnings", [])) errors.extend(extract_result.get("errors", [])) errors.extend(transform_result.get("errors", [])) if warnings:
logger.warning("Flow completed with warnings: %s", warnings) if errors: logger.warning("Flow completed with recoverable errors: %s", errors) return Completed( name="CompletedWithWarnings" if (warnings or errors) else "Completed", message=( f"Flow completed with {len(warnings)} warnings and " f"{len(errors)} recoverable errors" if (warnings or errors) else "Flow completed successfully" ), data={ "warnings": warnings, "errors": errors, "records_read": extract_result["records_read"], "records_written": transform_result["records_written"], }, )
Copy code
This gives you:

- a green flow run
- a visible warning-oriented state name/message
- structured metadata for later inspection

---

*What I’d recommend in most real projects*

*For tasks:*
- return structured data like:
  - `processed_count`
  - `failed_count`
  - `failures`
  - `warnings`
- only use `Completed(...)` if the UI signal matters

*For flows:*
- aggregate those task outputs
- log warnings prominently
- optionally return a final custom completed state like `CompletedWithWarnings`

---

*Practical result shape*

A good reusable shape is something like:
{ "ok": true, "warning_count": 3, "error_count": 1, "warnings": [...], "errors": [...], "metrics": {...} } ``` That tends to be easier to work with than trying to encode too much business meaning into Prefect states themselves. --- Important caveat If you return 
Completed(...)
, Prefect orchestration will treat that as success. So do not use this pattern if you need: - retries - failure notifications - downstream prevention on bad data In those cases, fail the task/flow and put the details in the exception, logs, or result data. --- My actual “best practice” summary - Business-level partial success → return structured results, optionally 
CompletedWithWarnings
- Operational failure → raise / fail the run - Need green but visible attention → 
Completed(name="CompletedWithWarnings", message=...)
- Need automation on warnings → emit logs/metrics or build automations around messages/tags/results, not around a special built-in warning state If you want, I can give you a production-ready helper pattern like a reusable 
WarningSummary
model plus a decorator/util for standardizing this across all flows.
c
@Marvin: How would these CompletedWithWarnings states render in the UI?
m
✅ thought for 7.4 seconds
They’d still render as Completed in terms of status category/color/behavior, because the underlying state type is still 
COMPLETED
. What changes is the display name/message. So if you do:
Copy code
from prefect.states import Completed

return Completed(
    name="CompletedWithWarnings",
    message="95 succeeded, 5 skipped"
)
in the UI you should expect roughly: - the run is shown as successful/completed - it uses the visual treatment of a completed run, not failed/warning-specific orchestration styling - the custom name 
CompletedWithWarnings
can appear as the state name - the message 
95 succeeded, 5 skipped
can appear in the run details / state details So operationally it is still “green,” just with more descriptive labeling. Important implication: if your team scans the UI primarily by color/status, this will not stand out the same way a failed run would. It’s more of a “green with text context” pattern, not a separate warning severity. If you want stronger visibility, common additions are: - log a warning in the flow/task logs - include warning counts in the run name or deployment naming conventions - use automations based on logs/events/results if needed - persist a structured summary in the result data If you'd like, I can show a small screenshot-equivalent mental model of where 
state type
, 
name
, and 
message
tend to show up in the Prefect UI/API, or give you a tiny flow you can run locally to see it.
c
@Marvin: Would the new State show up in filters or become a notification trigger option?
m
✅ thought for 6.9 seconds
Short answer: generally no, not as a new first-class state type. If you use:
Copy code
Completed(name="CompletedWithWarnings", ...)
the run’s type is still 
COMPLETED
. The custom 
name
is just labeling/context on that completed state. So in practice: - Filters based on state type/status → it will behave like 
COMPLETED
- Notification / automation triggers based on run state type → it will also behave like 
COMPLETED
- it does not create a brand-new canonical state category alongside 
FAILED
, 
CRASHED
, etc. So if you’re hoping for: - a dedicated UI filter like 
CompletedWithWarnings
- a built-in automation trigger like “when state becomes CompletedWithWarnings” that’s not the usual behavior. What you may be able to use, depending on where you're filtering/inspecting, is: - state name as metadata - state message - logs - returned/persisted result data - events/automations driven by other signals you emit Best practice if you need notifications for warnings: 1. Keep the flow/task 
COMPLETED
2. Emit an explicit warning signal elsewhere: - warning log line - custom event - structured result payload 3. Trigger alerting off that signal rather than expecting a new state class So the conceptual model is: - state type = orchestration behavior - state name/message = operator-facing annotation If you want, I can research the current Prefect Cloud automation/filter capabilities specifically around whether state names (not just types) are exposed in filters/triggers in the current UI/API.
n
it might be an interesting feature request to make an issue like:
allow UI to respect custom state names
there might be some interesting tidbits to solve, but OTOH seems achievable and useful
Open in Slack
PreviousNext
Powered by