I'd love to know what everyone is currently doing ...
# ask-community
a
I'd love to know what everyone is currently doing for event based flows (given that https://docs.prefect.io/core/PINs/PIN-14-Listener-Flows-2.html) is currently paused. I have a massive home grown job system that spans multiple runtimes all inside Kubernetes jobs that I would like to start porting to something like prefect so we can get a better handle on reliability and visibility. What I need to be able to do is start flows in response to end-user config changes. Ideally there is also a way to throttle or debounce based on parameters.
a
How would the end users provide the config changes? 1. Via some custom in-house built UI? If so, you could build an API with something like FastAPI or Flask, and ceate a flow run via API call using create_flow_run task. Using this task, you can set custom parameters, override run config, and more 2. If those users would have access to Prefect UI, then they could trigger flows directly from Prefect UI “run” option - this also allows to modify parameters and run configuration options such as amount of cpu or memory Overall, since creating a flow run is a matter of API call, you could combine Prefect with something like AWS Lambda to run flows based on various events - new S3 objects in your bucket, new rows in your DynamoDB, new message in your SNS topic - the possibilities here are endless. Here is a blog post that explains that: https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a
option #2 would be the easiest one, but option #3 with Lambda would be most flexible in terms of true event-driven workflows
a
Okay, probably easiest would be using
create_flow_run
from the other services.
👍 1