Hi all! Sharing some work on using Pub/Sub messages to trigger Prefect flows! I chose to use Memphis as a message broker. My use case was to gather all our automations in one tool, we use Prefect for our data processing pipelines, GitLab CI/CD, and other internal tools for databases, front-end, ect... Now we integrate with Memphis to handle all kind of events in one place and automate all executions from new data processed to ML model training and frontend updates! Code : https://discourse.prefect.io/t/using-a-message-broker-to-trigger-flow-runs/3485

awesome! thanks for sharing this curious if you've checked out the Events / webhooks system in prefect cloud at all? it seems like Memphis could also bring in some events that could be used to create webhooks - events in prefect cloud> if you were interested in doing more event-driven things in your prefect ecosystem!

Yes I did! I chatted a bit with Memphis' team, there is no native integration in Memphis to produce webhooks events right now, it can receive event from webhooks but not publish them. Although it's coming to the roadmap soon and hopefully will work without needing an intermediate server as I do today 🙂

This is awesome Emma, DMing you.