<@ULVA73B9P> Can you elaborate on `for_each` with ...
# ask-marvin
l
@Marvin Can you elaborate on
for_each
with an example event trigger? Per the docs, it says:
Evaluate the Automation separately for each distinct value of these labels on the resource. By default, labels refer to the primary resource of the triggering event. You may also refer to labels from related resources by specifying
related:<role>:<label>
. This will use the value of that label for the first related resource in that role.
So I guess the values can only be labels i.e. the primary or the related resource ids?
m
The
for_each
parameter in Prefect's event triggers allows you to evaluate the automation separately for each distinct value of specified labels on the resource. These labels typically refer to the primary resource of the triggering event, such as resource IDs or other identifying labels. ### Example Usage of
for_each
Suppose you have an event trigger that should respond to flow run events. You can use
for_each
to ensure that the automation is evaluated separately for each flow run ID. Here's a conceptual example:
Copy code
yaml
triggers:
  - type: event
    event: prefect.flow-run.Completed
    for_each:
      - prefect.resource.id
    actions:
      - type: run_flow
        parameters:
          flow_id: my_flow_id
          parameters:
            param1: value1
### Key Points: - Labels: The
for_each
parameter uses labels, which can be resource IDs or any other unique identifiers associated with the event. These labels help in executing the automation separately for each unique occurrence. - Primary and Related Resources: By default,
for_each
applies to the primary resource of the event. However, you can refer to related resources by specifying
related:<role>:<label>
, allowing for more granular control. For more details, you can refer to the Prefect documentation on custom triggers.
l
@Marvin Can you give an example where you use related resource instead ? Lets say the event was triggered from prefect webhook
m
When using a webhook to trigger events in Prefect, you can utilize related resources to evaluate automations separately for each distinct value. Here's an example scenario where you might use a related resource in the
for_each
parameter: ### Example Scenario Suppose you have a webhook that triggers an event every time a data model is updated. The webhook itself is a related resource in the events it produces. You can set up an automation to respond to these webhook events and evaluate the automation separately for each distinct model ID provided in the webhook's payload. ### Example Configuration
Copy code
yaml
triggers:
  - type: event
    event: webhook.model-updated
    for_each:
      - related:webhook:model_id
    actions:
      - type: run_flow
        parameters:
          flow_id: my_flow_id
          parameters:
            model_id: "{{ related.webhook.model_id }}"
### Key Points: - Webhook as a Related Resource: The webhook itself is added as a related resource on all events it produces. This allows you to refer to specific fields or identifiers from the webhook payload. - Dynamic Evaluation: By using
for_each
with
related:webhook:model_id
, the automation is evaluated separately for each unique
model_id
received from the webhook. This ensures that each model update is handled individually. For more details, you can refer to the Prefect documentation on webhook triggers.
l
@Marvin What would the webhook template look like in this case?
m
To create a webhook template in Prefect that handles dynamic information, such as model updates, you can use Jinja2 templating to extract and utilize values from the webhook's payload. Here's an example of how this might look: ### Webhook Template Example Imagine you have a webhook that receives POST requests with information about updated models. The request body includes fields like
model_id
and
friendly_name
. Here's how you might set up the webhook template:
Copy code
yaml
template:
  event: model-updated
  resource:
    id: "{{ body.model_id }}"
    type: model
    labels:
      - "model-updates"
  payload:
    model_id: "{{ body.model_id }}"
    friendly_name: "{{ body.friendly_name|default(body.model_id) }}"
### Key Points: - Dynamic Fields: The
model_id
and
friendly_name
are extracted from the request body using Jinja2 syntax. The
default
filter is used to handle missing values, substituting
model_id
if
friendly_name
is not provided. - Event and Resource Definition: The
event
and
resource
sections define the type of event and the resource it pertains to. You can use these fields to uniquely identify and categorize the events in Prefect. - Payload: Additional information can be included in the
payload
, which can be used by your automations or for debugging purposes. For more details, refer to the Prefect documentation on webhook triggers.
l
@Marvin Ahh but this is same as using the resource id/related resource id of the triggering event no? Since the trigger is executed for the corresponding event only once and not based on values of certain field. For eg this wouldn't be something like for each model_id in model_ids in a single invocation?
m
You're correct in noting that the
for_each
functionality, as described, evaluates automations separately for each distinct value of specified labels on the resource, typically focusing on unique identifiers like resource IDs. This means that the trigger is executed based on the distinct values of these identifiers, and not iteratively over a list of values within a single event payload. ### Clarification on `for_each`: - Single Invocation: The current
for_each
implementation in Prefect triggers separate automations for each unique label value from separate events, not for multiple values within a single event payload. - Distinct Events: It operates by evaluating distinct events, where each event might carry a unique identifier (like
model_id
), and the automation runs separately for each distinct event. ### Not Iterative Over Payload Lists: - If you want to iterate over a list of values contained within a single event payload (e.g., multiple
model_ids
in one webhook call), Prefect's current
for_each
mechanism isn't designed to handle that scenario directly. - You would need to handle such logic within the flow itself or by using additional scripting to split the event into multiple events, each with a single identifier. In essence, Prefect's
for_each
is centered around handling distinct events with unique identifiers, rather than iterating over a collection of values within a single event. If you need more complex iteration over payload data, you might need to consider handling it at the application level before triggering the events.