Thread
#prefect-community
    Joseph Ellis

    Joseph Ellis

    1 year ago
    And another one from me, on a completely different topic. It looks like Prefect has good support for AWS Lambda, which is awesome. However, I couldn’t see a way to specify the AWS endpoint to use, only the credentials etc. Am I wrong? Our use case for Prefect is to string a series of Lambdas together, and we’re using serverless framework (serverless offline) in our local development environment to run Lambdas on a localhost endpoint. It would be great if I could get Prefect local talking to my local Lambdas.
    Dylan

    Dylan

    1 year ago
    Hi @Joseph Ellis! Welcome to the community 👋 It sounds like you’re using Prefect Tasks to orchestrate Lambda function in a particular order and you’ve got that working nicely. Are you now trying to change what endpoint your Tasks attempt to communicate with depending on some sort of environment? (Dev vs. Prod, for example)
    Joseph Ellis

    Joseph Ellis

    1 year ago
    Hi Dylan, thanks for your response. My use-case at the moment is focussed on our local development environment.
    We’re able to run our Lambda functions on a local endpoint before deploying to AWS, using serverless offline.
    We can then invoke these lambdas using boto3, by passing in a specific endpoint_url to our boto3 client instantiation: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
    Dylan

    Dylan

    1 year ago
    Gotcha
    Joseph Ellis

    Joseph Ellis

    1 year ago
    I’m just trying to work out if there’s something I can do with the LambdaInvoke class that Prefect provides to allow me to point to this local endpoint.
    Dylan

    Dylan

    1 year ago
    So you’re looking for a way to configure the endpoint based on the context?
    I think you have a couple of options
    Joseph Ellis

    Joseph Ellis

    1 year ago
    Correct
    Dylan

    Dylan

    1 year ago
    You can use Parameters (https://docs.prefect.io/core/concepts/parameters.html) to pass a value into your Flow Run
    Parameters also have defaults
    When you create a schedule in Prefect Cloud, you can specify default parameters for a particular schedule
    (In the python code we would say that you can specify different default parameters for each Clock on your Schedule)
    Joseph Ellis

    Joseph Ellis

    1 year ago
    Apologies Dylan, this is specifically about over-riding the default endpoint_url that is passed to the lambda_client created here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/tasks/aws/lambda_function.py
    Dylan

    Dylan

    1 year ago
    Ahh okay
    If you want to add functionality to a particular Task in our Task Library, we’ll need to open an issue
    I can do that here or you’re welcome to write one up 😄
    Also, if you’re looking for something a bit more immediate, you can grab the functionality from this task and implement changes yourself in a custom Task
    Joseph Ellis

    Joseph Ellis

    1 year ago
    It’s alright, I’ve cracked it 🙂
    Using kwargs I can pass additional parameters to the boto3 client.
    Dylan

    Dylan

    1 year ago
    Excellent!
    😛arty-parrot: