<@ULVA73B9P> Pydantic validators don't seem to be ...
# ask-marvin
s
@Marvin Pydantic validators don't seem to be respected when running a prefect flow (from the UI), even with the 'validate parameters' box checked. The flow begins running, then it fails with
Process for flow run 'abc-123' exited with status code: 1
when ideally it would fail with the actual validation error, or even better, the form would not be submitted. Is this a known issue?
m
thought for 302.2 seconds
Short answer: this is expected behavior, not a bug in Pydantic itself. What’s happening - The “Validate parameters” checkbox in the UI performs schema-based validation only. It checks your parameters against the deployment’s parameter OpenAPI/JSON Schema and blocks submission if they don’t conform. It does not (and cannot) run your custom Pydantic validators in the UI. - Custom Pydantic (v2) validators run at execution time when the flow starts (Prefect 3.x’s
@flow
has
validate_parameters=True
by default). If validation fails, the run should enter a Failed state with a message containing the Pydantic error. In some views you may still see a generic “Process exited with status code: 1” log line, but the detailed reason should be in the flow run’s state message. How to get the form to block submission - Expose your constraints in the JSON schema so the UI can validate them pre-submit. In practice, that means using Field constraints or Annotated types so Prefect can derive them into the parameter schema:
Copy code
from typing import Annotated
from pydantic import BaseModel, Field
from prefect import flow

class Params(BaseModel):
    # UI can enforce these because they become JSON Schema constraints
    name: Annotated[str, Field(min_length=3)]
    age: Annotated[int, Field(ge=0, le=120)]

@flow
def my_flow(params: Params):
    ...
- If you’re using only custom validators like
@field_validator
, those will run at execution time, not in the UI. Where to find the actual validation error - Open the flow run page and check the latest state (Failed). Expand it to see the full state message. That message should include the Pydantic ValidationError details (e.g., which field failed and why). - You can also fetch it via the API/Client and inspect
flow_run.state.message
. If you only see “exited with status code: 1” - That’s usually a generic subprocess summary message. There should still be a Failed (or Crashed) state with a message that includes the validation error. If you don’t see it, that may be an issue worth investigating. - To help us reproduce, could you share: - Prefect version (assuming 3.x?), Pydantic version - How you’re launching (Cloud/Server, which worker type) - A minimal example of your parameter model (showing the validators) - A screenshot or copy of the run’s latest state message Notes - Prefect 3.x uses Pydantic v2 under the hood. Custom validators should be declared with
@field_validator
/
@model_validator
(not the v1
@validator
). - The UI can only enforce what’s present in the generated parameter schema. Using
Field(...)
constraints or
Annotated[...]
is the best way to make the form block invalid submissions.
s
@Marvin Uses a
@model_validator
, Prefect 3.x, Pydantic v2, ECS task, Prefect Cloud, the state message is the one I sent over
m
thought for 101.8 seconds
Thanks for the details — that helps. What you’re seeing breaks down into two parts: 1) Why the UI didn’t block submission - The “Validate parameters” checkbox only validates against the deployment’s generated JSON/OpenAPI schema. It cannot execute your Pydantic
@model_validator
at form time. - To make the UI block submission, encode those rules as JSON Schema constraints (e.g., via
Field(...)
constraints or
Annotated[...]
) so the schema reflects them. Example:
Copy code
from typing import Annotated
from pydantic import BaseModel, Field
from prefect import flow

class Params(BaseModel):
    x: Annotated[int, Field(ge=0)]  # UI can enforce this pre-submit

@flow
def my_flow(p: Params):
    ...
- If your deployment doesn’t have a parameter schema attached, the checkbox won’t have any effect. In Cloud, open the deployment and confirm a parameter schema is present. If it’s missing, redeploy the flow so the schema is generated from the function signature. 2) Why the run only shows “exited with status code: 1” - Pydantic v2 validators (including
@model_validator
) do run at execution time in Prefect 3.x. If validation fails, Prefect should put the run into a Failed state with a message like “Validation of flow parameters failed with error: …” including the Pydantic details. - If you only see “Process for flow run ... exited with status code: 1”, that typically indicates the process crashed before Prefect could attach the detailed failure message — you’ll often see a Crashed state instead of Failed in those cases. - This isn’t a known widespread issue specific to
@model_validator
, but it can happen in remote infrastructure (ECS) if the process terminates early (e.g., before Prefect’s engine handles the exception). We’d like to confirm whether your run state is Failed or Crashed. Quick checks to narrow this down - In the run’s page, check the latest state type: - If it’s Failed, expand the state to see the full state message — that’s where the Pydantic error should appear. - If it’s Crashed, Prefect likely didn’t get a chance to write the validation error. Check the ECS task logs (CloudWatch) for the detailed traceback. - Enable more logging to surface the validation message: - Add env var
PREFECT_LOGGING_LEVEL=DEBUG
to the ECS task definition. - Try a minimal reproducer:
Copy code
from pydantic import BaseModel, model_validator
from prefect import flow

class P(BaseModel):
    x: int

    @model_validator(mode="after")
    def check_x(self):
        if self.x < 0:
            raise ValueError("x must be >= 0")
        return self

@flow
def f(p: P):
    return p.x
Deploy and run with
p={"x": -1}
. Expected: Failed state with a message including the Pydantic error. If the minimal example still only shows the generic “status code: 1” in Cloud for ECS runs, that points to an execution-time crash before Prefect writes the Failed state (possibly an infra/logging nuance). In that case, please share: - Exact Prefect version (3.x.y) and the worker/flow image tag in your ECS task - Whether the run state is Failed or Crashed - The snippet of your
@model_validator
and flow signature - A snippet from the ECS task logs, if available Workarounds/next steps - To prevent invalid submissions from the UI, expose constraints in the schema with `Field(...)`/`Annotated[...]` as shown above. - To make the error visible to users, you can also add a Cloud Automation for “Flow run enters Failed” and include
{{ flow_run.state_message }}
in the notification while we confirm what’s happening on ECS. If you can provide the above details or a small repro, I’m happy to try to reproduce and file an issue if it turns out the engine isn’t surfacing the Pydantic error on ECS as expected.
s
@Marvin It is a failed state, not a crashed one
m
thought for 44.1 seconds
Got it — thanks. Since it’s a Failed (not Crashed) state, I’d normally expect the Pydantic ValidationError details to be present in the run’s state message. Seeing only “Process for flow run ... exited with status code: 1” suggests the failure is happening outside Prefect’s parameter‑validation path or very early in the run lifecycle. A couple targeted checks to pinpoint this: - Confirm your flow signature uses the Pydantic model type directly and relies on Prefect’s parameter validation: - Good:
@flow()
then
def my_flow(p: MyModel): ...
with
@model_validator
on
MyModel
(and don’t set
validate_parameters=False
). - Less ideal (and can lead to generic messages): accepting
dict
(or untyped) and then constructing
MyModel
later inside the flow or in a task. - Make sure the model isn’t being constructed at import time. Any exception raised before the engine enters the run context (e.g., module-level code) can result in a generic “status code: 1” message. - On the flow run page, open: - States panel: expand the latest Failed state and copy the full “State message”. - Logs tab: copy the first error/traceback line you see around the time the run starts. - ECS task logs (CloudWatch): check for a traceback right after the task starts. If you can share a minimal repro, I can try to replicate exactly what you’re seeing. For example, this should produce a Failed state with a clear validation message:
Copy code
from pydantic import BaseModel, model_validator
from prefect import flow

class P(BaseModel):
    x: int

    @model_validator(mode="after")
    def check(self):
        if self.x < 0:
            raise ValueError("x must be >= 0")
        return self

@flow  # validate_parameters defaults to True in Prefect 3.x
def f(p: P):
    return p.x
Deploy and run with
p={"x": -1}
. Expected: a Failed state with a message indicating the validation error. If this minimal case still shows only “status code: 1” in the state message on ECS, that points to the error being swallowed before the engine records details — and I’ll open an issue for us to fix. In parallel, if you want the UI form to block submission, encode the rules as JSON Schema constraints so the “Validate parameters” checkbox can enforce them pre-submit (e.g.,
Field(min_length=..., ge=..., le=...)
or
Annotated[...]
), since the UI cannot run
@model_validator
at form time. If you can share: - Your flow signature (just the def line) and the Pydantic model with the
@model_validator
- Whether
validate_parameters
is left as default - A snippet of the run’s full Failed state message and the first relevant log line …I’ll help chase this down and, if needed, file a bug so those validation details reliably show up in the Failed state.
s
@Marvin You're right that the parameter validation error exists in the ECS log events. But one of the last logs before stopping the runner is
18:37:46.747 | *ERROR*   | prefect.flow_runs.runner - Process for flow run 'augmented-mussel' exited with status code: 1
m
thought for 83.1 seconds
Thanks — that lines up with how the runner works. What you’re seeing: - The ECS task (child process) is doing parameter validation, logging the detailed Pydantic error, and setting the run to Failed with that message. That’s the important/authoritative state. - Separately, the parent process notices the child exited non‑zero and logs the generic line: 183746.747 | ERROR | prefect.flow_runs.runner - Process for flow run 'augmented-mussel' exited with status code: 1 - After that log, the parent attempts to mark the run as Crashed, but since the child already set a terminal Failed state, the server rejects the crash proposal. The Failed state with the detailed validation message remains intact. So the generic “exited with status code: 1” line is expected noise from the parent runner whenever the subprocess exits non‑zero — even in “good” failures like validation errors. It does not replace your detailed failure reason. If this is confusing in your observability: - Rely on the run’s latest state and its message as the source of truth. - In notifications/automations, include the state message (e.g., {{ flow_run.state_message }}) so users see the Pydantic details, not the generic exit line. - If you control the worker image, you can reduce noise by raising the logger level for the parent runner component. For example, set PREFECT_LOGGING_LEVEL=INFO (and optionally add a logger filter for prefect.flow_runs.runner to suppress the exit-code line). If you want, I can share a small logging config snippet to filter that line. If you ever see the generic exit log and the final state is Crashed (with no detailed message) for a parameter validation failure, that would point to the child being unable to persist the Failed state — that’d be worth reporting. But given yours is a proper Failed state with a validation message in ECS logs, you’re on the expected path.