Thread
#prefect-community
    s

    Scott Vermillion

    1 year ago
    Hello Prefect Land! Simple request. I have a notification (new file upload) set up in AWS S3 that goes to SNS. From there, it branches to send an e-mail and also kick off a lambda function that kicks off a flow in Prefect. When I receive the e-mail, I can see the entire file path in S3. How can I get that same info over to my Prefect flow so I can parameterize? I hope I’m making sense… Best All!
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @Scott Vermillion, you’ll likely need to pass it as a Parameter. How do you kick off the Flow run? Could you insert it there?
    s

    Scott Vermillion

    1 year ago
    Hi @Kevin Kho. Yes, I do want to pass the file name/path as a parameter to the flow. What I have right now is a lambda function in AWS that kicks over to Prefect Cloud when a new file gets dumped. When I get an e-mail notification from AWS, I can see the entire path of the new file. But I’m not sure how to capture that info and send that into Prefect Cloud? To do all of this, I followed a blog post that I believe was from your CTO. https://medium.com/the-prefect-blog/event-driven-workflows-with-aws-lambda-2ef9d8cc8f1a Hopefully I’m not missing something obvious, but I’m not clear where I could capture that filename and send it into Cloud. But that info is somewhere inside of AWS, because when its SNS service sends me an e-mail at the same time it kicks off the lambda, I get that piece of intelligence.
    Kevin Kho

    Kevin Kho

    1 year ago
    Wouldn’t the snippet there work? Something like:
    # pass the full event
    inputs['parameters'] = dict(event=event)
    
    # or just the bucket name
    inputs['parameters'] = dict(bucket_name=event['Records'][0]['s3']['bucket']['name'])
    You would just need to get the s3 file instead of the bucket so it becomes:
    inputs['parameters'] = dict(file_path=event['Records'][0]['s3']['object']['key'])
    or something like that? I don’t know what the full event looks like so this is just a guess based on this . And then on the Prefect side, you would accept it as
    with Flow(...) as flow:
         file_path = Parameter("file_path")
    And then
    file_path
    will take the value for the file_path you passed in
    s

    Scott Vermillion

    1 year ago
    Oh, yes, that looks like exactly right. Sorry, I haven’t done anything with parameters yet (still in Prefect newb stage), so I think I did indeed miss something quite obvious. Thanks as always, @Kevin Kho!