i

    itay livni

    1 year ago
    Hi - I have a flow that runs under a minute and is part of another flow. I was thinking of wrapping the flow in an aws lambda to speed up the overall processing of the larger flow. Any thoughts?
    Kevin Kho

    Kevin Kho

    1 year ago
    Hey @itay livni, how long is the larger flow? You could but I’m not sure it’s worth the lambda dependency unless you have other lambda components already.
    i

    itay livni

    1 year ago
    Small - I am trying to work around large cold start times. Actually the thought is to use lambda for the second flow also. Basically I want to use prefect to orchestrate some lamdas so I don’t need to spin up large Dask clusters. The first flow is mapped over, the second flow is reduce.
    Kevin Kho

    Kevin Kho

    1 year ago
    Wouldn't lambda also have a cold start depending how often you run the flow?
    i

    itay livni

    1 year ago
    It would but because it is mapped over and iterated over 10 -20 times I think it might be faster by treating it as an "app"
    @Kevin Kho If I were to wrap a prefect flow in a lambda - What agent would I use? Local. I have a small ec2 instance in the cloud where I host the prefect server.
    Kevin Kho

    Kevin Kho

    1 year ago
    Yes local