Laxmi
02/27/2025, 6:45 AMfor_each
with an example event trigger? Per the docs, it says:
Evaluate the Automation separately for each distinct value of these labels on the resource. By default, labels refer to the primary resource of the triggering event. You may also refer to labels from related resources by specifyingSo I guess the values can only be labels i.e. the primary or the related resource ids?. This will use the value of that label for the first related resource in that role.related:<role>:<label>
Marvin
02/27/2025, 6:45 AMfor_each
parameter in Prefect's event triggers allows you to evaluate the automation separately for each distinct value of specified labels on the resource. These labels typically refer to the primary resource of the triggering event, such as resource IDs or other identifying labels.
### Example Usage of for_each
Suppose you have an event trigger that should respond to flow run events. You can use for_each
to ensure that the automation is evaluated separately for each flow run ID. Here's a conceptual example:
yaml
triggers:
- type: event
event: prefect.flow-run.Completed
for_each:
- prefect.resource.id
actions:
- type: run_flow
parameters:
flow_id: my_flow_id
parameters:
param1: value1
### Key Points:
- Labels: The for_each
parameter uses labels, which can be resource IDs or any other unique identifiers associated with the event. These labels help in executing the automation separately for each unique occurrence.
- Primary and Related Resources: By default, for_each
applies to the primary resource of the event. However, you can refer to related resources by specifying related:<role>:<label>
, allowing for more granular control.
For more details, you can refer to the Prefect documentation on custom triggers.Laxmi
02/27/2025, 6:54 AMMarvin
02/27/2025, 6:54 AMfor_each
parameter:
### Example Scenario
Suppose you have a webhook that triggers an event every time a data model is updated. The webhook itself is a related resource in the events it produces. You can set up an automation to respond to these webhook events and evaluate the automation separately for each distinct model ID provided in the webhook's payload.
### Example Configuration
yaml
triggers:
- type: event
event: webhook.model-updated
for_each:
- related:webhook:model_id
actions:
- type: run_flow
parameters:
flow_id: my_flow_id
parameters:
model_id: "{{ related.webhook.model_id }}"
### Key Points:
- Webhook as a Related Resource: The webhook itself is added as a related resource on all events it produces. This allows you to refer to specific fields or identifiers from the webhook payload.
- Dynamic Evaluation: By using for_each
with related:webhook:model_id
, the automation is evaluated separately for each unique model_id
received from the webhook. This ensures that each model update is handled individually.
For more details, you can refer to the Prefect documentation on webhook triggers.Laxmi
02/27/2025, 6:58 AMMarvin
02/27/2025, 6:58 AMmodel_id
and friendly_name
. Here's how you might set up the webhook template:
yaml
template:
event: model-updated
resource:
id: "{{ body.model_id }}"
type: model
labels:
- "model-updates"
payload:
model_id: "{{ body.model_id }}"
friendly_name: "{{ body.friendly_name|default(body.model_id) }}"
### Key Points:
- Dynamic Fields: The model_id
and friendly_name
are extracted from the request body using Jinja2 syntax. The default
filter is used to handle missing values, substituting model_id
if friendly_name
is not provided.
- Event and Resource Definition: The event
and resource
sections define the type of event and the resource it pertains to. You can use these fields to uniquely identify and categorize the events in Prefect.
- Payload: Additional information can be included in the payload
, which can be used by your automations or for debugging purposes.
For more details, refer to the Prefect documentation on webhook triggers.Laxmi
02/27/2025, 7:04 AMMarvin
02/27/2025, 7:04 AMfor_each
functionality, as described, evaluates automations separately for each distinct value of specified labels on the resource, typically focusing on unique identifiers like resource IDs. This means that the trigger is executed based on the distinct values of these identifiers, and not iteratively over a list of values within a single event payload.
### Clarification on `for_each`:
- Single Invocation: The current for_each
implementation in Prefect triggers separate automations for each unique label value from separate events, not for multiple values within a single event payload.
- Distinct Events: It operates by evaluating distinct events, where each event might carry a unique identifier (like model_id
), and the automation runs separately for each distinct event.
### Not Iterative Over Payload Lists:
- If you want to iterate over a list of values contained within a single event payload (e.g., multiple model_ids
in one webhook call), Prefect's current for_each
mechanism isn't designed to handle that scenario directly.
- You would need to handle such logic within the flow itself or by using additional scripting to split the event into multiple events, each with a single identifier.
In essence, Prefect's for_each
is centered around handling distinct events with unique identifiers, rather than iterating over a collection of values within a single event. If you need more complex iteration over payload data, you might need to consider handling it at the application level before triggering the events.