Hi Team, Appreciate your help with this.
How to effectively trigger the flow execution (create a flow run) when file arrives to S3 Bucket?
I am thinking to leverage the event from S3 which then triggers lambda function and aws lambda function then calls the client.create_flow_run_from_deployment?
Is that the right approach? Should I be able to call the create_flow_run_from_deployment from lambda given the limitation on library size? Or I should be looking into Rest API?
Are there any other-ways to start the flow I should be exploring?
Could you please point to some samples?
if you’re already in the context of a lambda function you don’t need webhooks imho since you can control 100% of the behavior, so just make a REST API call to invoke the deployment. The disadvantage of this is that it creates a tight coupling between the lambda and the delpoyment. the disadvantage of webhooks is that they pass through the public internet with no inherent security.
there is also a conceptual alternative (if this type of scenario is a common pattern and you’d like to have a lot of different triggers like these, or - you prefer to work in batch (and not react to every single file arrival immediately) - where instead of triggering the deployment directly, you simply push these events to some kind of queue (SQS, kafka, or anything else) and then consume the queue in a periodically/continuously running flow