https://prefect.io logo
#prefect-community
Title
# prefect-community
j

Joseph Ellis

03/29/2021, 3:39 PM
And another one from me, on a completely different topic. It looks like Prefect has good support for AWS Lambda, which is awesome. However, I couldn’t see a way to specify the AWS endpoint to use, only the credentials etc. Am I wrong? Our use case for Prefect is to string a series of Lambdas together, and we’re using serverless framework (serverless offline) in our local development environment to run Lambdas on a localhost endpoint. It would be great if I could get Prefect local talking to my local Lambdas.
👀 1
d

Dylan

03/29/2021, 3:50 PM
Hi @Joseph Ellis! Welcome to the community 👋 It sounds like you’re using Prefect Tasks to orchestrate Lambda function in a particular order and you’ve got that working nicely. Are you now trying to change what endpoint your Tasks attempt to communicate with depending on some sort of environment? (Dev vs. Prod, for example)
j

Joseph Ellis

03/29/2021, 3:50 PM
Hi Dylan, thanks for your response. My use-case at the moment is focussed on our local development environment.
We’re able to run our Lambda functions on a local endpoint before deploying to AWS, using serverless offline.
We can then invoke these lambdas using boto3, by passing in a specific endpoint_url to our boto3 client instantiation: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
d

Dylan

03/29/2021, 3:54 PM
Gotcha
j

Joseph Ellis

03/29/2021, 3:55 PM
I’m just trying to work out if there’s something I can do with the LambdaInvoke class that Prefect provides to allow me to point to this local endpoint.
d

Dylan

03/29/2021, 3:56 PM
So you’re looking for a way to configure the endpoint based on the context?
I think you have a couple of options
j

Joseph Ellis

03/29/2021, 3:57 PM
Correct
d

Dylan

03/29/2021, 3:57 PM
You can use Parameters (https://docs.prefect.io/core/concepts/parameters.html) to pass a value into your Flow Run
Parameters also have defaults
When you create a schedule in Prefect Cloud, you can specify default parameters for a particular schedule
(In the python code we would say that you can specify different default parameters for each Clock on your Schedule)
j

Joseph Ellis

03/29/2021, 3:58 PM
Apologies Dylan, this is specifically about over-riding the default endpoint_url that is passed to the lambda_client created here: https://github.com/PrefectHQ/prefect/blob/master/src/prefect/tasks/aws/lambda_function.py
d

Dylan

03/29/2021, 3:59 PM
Ahh okay
If you want to add functionality to a particular Task in our Task Library, we’ll need to open an issue
I can do that here or you’re welcome to write one up 😄
Also, if you’re looking for something a bit more immediate, you can grab the functionality from this task and implement changes yourself in a custom Task
j

Joseph Ellis

03/29/2021, 4:08 PM
It’s alright, I’ve cracked it 🙂
Using kwargs I can pass additional parameters to the boto3 client.
d

Dylan

03/29/2021, 4:09 PM
Excellent!
🦜
2 Views