Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Prefect Community

Hello everyone :wave:
I'm curious if anyone triggers prefect flows programmatically based on receiving messages. I'm particularly interested in AWS SQS / SNS events.
Are there any good design patterns there or convenient integrations?

Hi Fabien! Hopefully this helps with scoping out your work, but here are two examples for you to leverage:
• *<https://docs.prefect.io/v3/tutorials/ml|Train a machine learning model>*
• *<https://github.com/PrefectHQ/prefect-demos/tree/main/flows/aws/datalake|AWS datalake example>*
Both examples make use of AWS EventBridge and Prefect webhooks to emit events to your Prefect workspace. In turn the events can be used to trigger automations that kick off downstream flows. I imagine something similar could be done with AWS SNS. In short, the order of operations looks like this for the examples I shared:
S3 file created :arrow_right: EventBridge Rule :arrow_right: Prefect Webhook :arrow_right: Automation :arrow_right: Kick off Flow run

Amazing, thank you for sharing. I'll check those examples out :pray: