@Marvin on prefect 3, how to match a custom event with its flow run that emitted it, on a trigger?

To match a custom event with its flow run that emitted it on a trigger in Prefect 3.x, you can use Prefect's event triggers. Here is a concise way to do it: 1. Define Triggers in YAML: An event trigger can be set in the

prefect.yaml

file of your deployment. You can define the events to listen for and specify parameters based on the event data. Example from the Prefect documentation: - Ensure triggers are defined in your deployment YAML configuration:

yaml
     deployments:
     - name: my-deployment
       entrypoint: path/to/flow.py:decorated_fn
       work_pool:
         name: my-work-pool
       triggers:
       - type: event
         match:
           prefect.resource.id: my.external.resource
         expect:
         - external.resource.pinged

2. Use Matching: Utilize the

match

field to filter only for events relevant to specific flow runs or resources. You can use this field to filter events based on criteria like the flow run name or deployment ID. 3. Access Flow Run Information: Use the

prefect.runtime.flow_run

module to get runtime information about the flow run, such as parameters or the flow name, to construct names or fetch specific data associated. 4. Monitor Events: Set expectations in the trigger configuration using the

expect

field to specify what events count toward influencing the trigger. You can find more details about event triggers and matching on the Prefect 3.0 documentation for custom triggers and how to define them. For deeper insights, further exploration may be required as community discussions, such as on GitHub, suggest enhancements around the runtime evaluation of template variables may influence functionality. If you are encountering errors or specific issues, checkingPrefect GitHub issues may also help, as they might have discussions on related improvements or bug fixes. For instance, Lazy templating for deployments in prefect.yaml documents some enhancements around deployment time evaluations. Feel free to ask if you need more specific guidance or have other questions!

@Marvin

triggerOnIngestion = DeploymentEventTrigger(
    name="trigger-on-ingestion",
    match={"prefect.resource.id": [ "prefect.flow-run.*" ]},
    posture="Reactive",
    threshold=1,
    after=[
        ingestion_event
    ],
    expect=[
        "prefect.flow-run.Completed"
    ],
    for_each=[
        "related:flow-run:prefect.resource.id"
    ],
    within=600
)

this isn't working to match both on the for_each

It looks like you're trying to use a

DeploymentEventTrigger

to match custom events with their associated flow runs using the

for_each

and

match

fields. Here are some key points to help ensure the setup functions correctly: 1. Declare Related Resources: In Prefect, when using event triggers, make sure that the resources and related items are correctly defined. The

match_related

fields are crucial for targeting events that pertain to specific resources in your flow run or deployment scenario. This involves specifying resources like

prefect.resource.id

, which should correctly reference your desired flow runs or other relevant entities. 2. Correct Filtering: Ensure that the

match

field is accurately filtering the desired events. The "prefect.resource.id" should be mapped correctly to the events you are targeting, such as specific flow runs. Check that the wildcard usage like "prefect.flow-run.*" is appropriate for the granularity you're hoping to achieve. 3. Iterating Over Related Resources: With

for_each

, ensure that you're iterating over valid resource lists that are being emitted during the event. The parameter

related:flow-run:prefect.resource.id

must be available and valid in the event context. 4. Consult Documentation for Examples: You can use examples from Prefect documentation to guide your setup. Explore how compound or sequence triggers integrate underlying triggers to ensure that all needed conditions are properly evaluated. 5. Check for Other Issues: If the above doesn't resolve the issue, review the specific event context being emitted during your flow runs to ensure that all necessary properties are present and correct for evaluation. For more detailed examples, you might find the Prefect 3.0 documentation on custom triggers useful. The documentation elaborates on how to precisely scaffold event conditions and the interplay between different events. If you continue to experience difficulties, verifying against your specific Prefect setup, or exploring community discussions could uncover new insights.