<@ULVA73B9P> I have this trigger in prefect.yaml: ...
# ask-marvin
f
@Marvin I have this trigger in prefect.yaml:
Copy code
triggers:
      - name: dlt.tables.load events
        type: event
        enabled: true
        match:
          prefect.resource.id: sync.dlt
        expect:
          - dlt.tables.load.*
        parameters:
          table_models:
            __prefect_kind: json
            json:
              template: '{{ event.payload.table_models | tojson }}'
              __prefect_kind: jinja
          reset:
            template: '{{ event.payload.reset | default(false) }}'
            __prefect_kind: jinja
by my automation is failing with:
Copy code
"payload": {
    "action_index": 0,
    "action_type": "run-deployment",
    "invocation": "068ff732-8ae6-7160-8000-6681938dfa3c",
    "reason": "Validation error occurred for 'run-deployment' - Error creating flow run: Validation failed. Failure reason: 'table_models' is a required property",
    "validation_error": "Error creating flow run: Validation failed. Failure reason: 'table_models' is a required property",
    "status_code": 409
  },
why?
m
thought for 409.8 seconds
Short answer: your trigger fired, but the templating for
table_models
did not resolve to a value, so Prefect tried to create a flow run without a
table_models
parameter. Since your flow declares
table_models
as a required parameter, validation failed with “'table_models' is a required property”. Common causes: - The triggering event payload does not contain
payload.table_models
- The Jinja template renders nothing/undefined - The JSON wrapper fails to parse because the template didn’t yield valid JSON What to check and how to harden it: 1) Verify the actual event payload - Can you share a representative event payload for
dlt.tables.load.*
? Specifically, does it include
payload.table_models
and
payload.reset
? - You can temporarily add a debug parameter to inspect what the template sees:
Copy code
parameters:
  debug_payload:
    __prefect_kind: jinja
    template: "{{ event.payload | tojson }}"
2) Make the template robust with defaults If
table_models
may be absent sometimes, ensure it still produces valid JSON:
Copy code
triggers:
  - name: dlt.tables.load events
    type: event
    enabled: true
    match:
      prefect.resource.id: sync.dlt
    expect:
      - dlt.tables.load.*
    parameters:
      table_models:
        __prefect_kind: json
        json:
          template: "{{ event.payload.table_models | default({}) | tojson }}"
          __prefect_kind: jinja
      reset:
        template: "{{ event.payload.reset | default(false) }}"
        __prefect_kind: jinja
- Here, if
event.payload.table_models
is missing, it becomes
{}
instead of missing, so the parameter is present and valid. 3) Confirm your flow’s parameter name matches exactly - Ensure your flow’s signature uses
table_models
(and not a different name or nesting). Why this happens - In Prefect 3 automations, parameter values are rendered via Jinja at automation execution time. If a template renders to nothing or invalid JSON, the resulting parameter may be omitted entirely. Since your flow requires
table_models
, the creation of the flow run fails. Relevant docs - Access parameters in templates - Creating deployment triggers - Event triggers - Events - prefect.yaml format If you can paste one of the actual
dlt.tables.load.*
events (or the UI “Event” JSON), I can help adjust the template precisely.
f
@Marvin The event has
Copy code
"payload": {
    "table_models": [
      {
        "name": "adress",
       .. ommitted for privacy
      },
]
}
m
thought for 62.1 seconds
Thanks — that helps. Given that your payload can include table_models, the most likely reason for “'table_models' is a required property” is that at least one of the events matched by your wildcard (
dlt.tables.load.*
) did not include
payload.table_models
. When the template renders to nothing/undefined, Prefect ends up trying to create a flow run without the
table_models
parameter and validation fails. Two solid fixes: 1) Narrow the events you accept to those that always include table_models For example, if only “completed” events have the field:
Copy code
triggers:
  - name: dlt.tables.load events
    type: event
    enabled: true
    match:
      prefect.resource.id: sync.dlt
    expect:
      - dlt.tables.load.completed
    parameters:
      table_models:
        __prefect_kind: json
        json:
          template: "{{ event.payload.table_models | tojson }}"
          __prefect_kind: jinja
      reset:
        template: "{{ event.payload.reset | default(false) }}"
        __prefect_kind: jinja
2) Add a defensive default so the parameter is always present If an occasional event lacks the field, provide a safe default (e.g., empty list or dict) so the parameter is not dropped:
Copy code
triggers:
  - name: dlt.tables.load events
    type: event
    enabled: true
    match:
      prefect.resource.id: sync.dlt
    expect:
      - dlt.tables.load.*
    parameters:
      table_models:
        __prefect_kind: json
        json:
          template: "{{ (event.payload.table_models | default([])) | tojson }}"
          __prefect_kind: jinja
      reset:
        template: "{{ event.payload.reset | default(false) }}"
        __prefect_kind: jinja
Optional: add a guard to only run when the field exists (if you’re comfortable customizing actions): - Set an action-level
if
on the run-deployment action:
if: "{{ event.payload.table_models is defined }}"
If you can paste an example of the failing event (the one that triggered the 409), I can confirm which variant is missing
table_models
and suggest the exact
expect
filter or condition. Docs for reference: - Accessing parameters in templates: docs - Creating deployment triggers: docs - Event triggers and events: event triggers, events
f
@Marvin Can you elaborate on this part "- Set an action-level
if
on the run-deployment action:
if: "{{ event.payload.table_models is defined }}"
m
thought for 219.7 seconds
Great question — and I need to correct myself here. There isn’t an action-level if for deployment triggers defined in prefect.yaml. In Prefect 3, the trigger determines when it fires (via match/expect/after/etc.), and when it does, the associated run-deployment action executes. There’s no supported if key you can put alongside parameters to conditionally skip the action at runtime. Ways to achieve the same outcome: - Narrow the events you accept: Set expect to the specific event name(s) that always include payload.table_models (e.g., dlt.tables.load.completed instead of dlt.tables.load.*). - Make the parameter robust with defaults: Ensure the template always yields valid JSON so the parameter is present, e.g. default to []:
Copy code
parameters:
  table_models:
    __prefect_kind: json
    json:
      template: "{{ (event.payload.table_models | default([])) | tojson }}"
      __prefect_kind: jinja
  reset:
    template: "{{ event.payload.reset | default(false) }}"
    __prefect_kind: jinja
- Make the flow parameter optional: If you control the flow signature, give table_models a default (e.g., None or []), then early-return if it’s missing. That avoids validation failures entirely. - Gate inside the flow: Add a boolean parameter like skip_if_missing and compute it from the event, then early-return in the flow. For example: - In the trigger:
Copy code
parameters:
      skip_if_missing:
        template: "{{ event.payload.table_models is not defined }}"
        __prefect_kind: jinja
- In the flow:
Copy code
if skip_if_missing:
        return
This doesn’t prevent the run from being created, but it prevents unnecessary work. Docs for reference: - Creating deployment triggers: docs - Access parameters in templates: docs - Event triggers: concepts and events If you share the exact event names that include table_models versus those that don’t, I can suggest a precise expect filter (or a robust default) tailored to your events.
f
@Marvin Are you sure that this.
Copy code
parameters:
          table_models:
            __prefect_kind: json
            json:
              template: '{{ event.payload.table_models | tojson }}'
              __prefect_kind: jinja
          reset:
            template: '{{ event.payload.reset | default(false) }}'
            __prefect_kind: jinja
m
thought for 63.2 seconds
Mostly yes for table_models, but I’d tweak two things: - table_models: Your pattern is correct for producing a real list/dict value from the event using nested jinja + json. I’d add a default to avoid missing-field failures:
Copy code
table_models:
  __prefect_kind: json
  json:
    __prefect_kind: jinja
    template: "{{ (event.payload.table_models | default([])) | tojson }}"
- reset: As written, this will render a string ("false"/"true") from Jinja, not a boolean. If your flow expects a bool, wrap it in json as well so it’s parsed to a real boolean:
Copy code
reset:
  __prefect_kind: json
  json:
    __prefect_kind: jinja
    template: "{{ event.payload.reset | default(false) | tojson }}"
Why: Jinja templates render text. The outer __prefect_kind: json tells Prefect to parse the rendered text as JSON, yielding the correct Python types (list/dict/bool). Without the json wrapper, the value is just a string. Docs for reference: - Access parameters in templates: docs - Creating deployment triggers: docs If you keep seeing “table_models is a required property,” it means at least one event that matches dlt.tables.load.* didn’t include that field. Either narrow expect to specific event names that always include it, or keep the default([]) shown above.
f
@Marvin but can I really match and mix jinja and json inputs?
m
thought for 31.4 seconds
Yes — mixing them is both supported and the recommended way to get correctly typed values from templates. How it works - __prefect_kind: jinja tells Prefect to render a Jinja template to a string. - __prefect_kind: json tells Prefect to parse a string as JSON to produce real Python types (list, dict, bool, int, etc.). - Combining them lets you render dynamic content with Jinja, then parse it into the right type with JSON. Typical patterns - Structured data (list/dict) from event payload:
Copy code
table_models:
  __prefect_kind: json
  json:
    __prefect_kind: jinja
    template: "{{ (event.payload.table_models | default([])) | tojson }}"
- Boolean with a safe default:
Copy code
reset:
  __prefect_kind: json
  json:
    __prefect_kind: jinja
    template: "{{ event.payload.reset | default(false) | tojson }}"
Pitfalls to avoid - If you omit the outer json wrapper, the result is a string (e.g., "false" instead of the boolean false). - Always use the tojson filter inside the Jinja template when producing non-string values. It guarantees valid JSON for the json parser to consume. - Don’t double-encode (e.g., avoid tojson twice); that will yield a JSON string containing quotes instead of the desired type. Docs for reference: - Access parameters in templates - Creating deployment triggers
f
@Marvin nice, but my parameter is a pydantic type.
m
thought for 157.6 seconds
Yes. For a Pydantic-typed parameter, you just need the rendered value to be a regular Python object that matches the model’s schema. The Jinja+JSON pattern is exactly how to get there: Jinja renders text, then the JSON wrapper parses that text into the correctly typed Python structure that Pydantic can validate and coerce. Two common cases: - Your parameter is a list of Pydantic models (e.g., list[TableModel]) - Pass a list of dicts (from the event) and let Pydantic do the coercion:
Copy code
parameters:
    table_models:
      __prefect_kind: json
      json:
        __prefect_kind: jinja
        template: "{{ (event.payload.table_models | default([])) | tojson }}"
    reset:
      __prefect_kind: json
      json:
        __prefect_kind: jinja
        template: "{{ event.payload.reset | default(false) | tojson }}"
- Your parameter is a single Pydantic model (e.g., class LoadSpec(BaseModel): table_models: list[TableModel]; reset: bool = False) - Build the full object shape in Jinja, then JSON-parse it:
Copy code
parameters:
    load_spec:
      __prefect_kind: json
      json:
        __prefect_kind: jinja
        template: "{{ {'table_models': (event.payload.table_models | default([])), 'reset': (event.payload.reset | default(false))} | tojson }}"
Tips for Pydantic types: - Make sure your template yields valid JSON for the model’s expected field types. The outer json wrapper ensures booleans, numbers, lists, dicts, etc. become real Python types. - If some fields are optional in your model, provide sensible defaults in the template to avoid missing-field validation errors. - For fields like datetime/UUID/Decimal, ensure the event provides values in formats Pydantic can parse (e.g., ISO 8601 strings for datetimes). - Avoid double-encoding (don’t apply tojson twice). If you share your exact Pydantic model definition and the flow signature, I can tailor the template to match it precisely.