https://prefect.io logo
Title
t

Tomás Emilio Silva Ebensperger

07/06/2022, 8:48 PM
I am running prefect 2.0 on aws lambda (runtime python3.9) but i get an error
"Orion requires sqlite >= 3.24.0 but we found version 3.7.17"
Does anyone know a workaround? i
1
k

Kevin Kho

07/06/2022, 8:52 PM
You can try upgrading python version which will upgrade sqlite
t

Tomás Emilio Silva Ebensperger

07/06/2022, 8:53 PM
i think their 3,9 runtime is their latest and they don't have a newer one
k

Kevin Kho

07/06/2022, 9:06 PM
Will check with the team on this
t

Tomás Emilio Silva Ebensperger

07/06/2022, 9:08 PM
thank you
z

Zanie

07/06/2022, 9:16 PM
I don’t think we can do anything about this at this time
We’ve got a firm SQLite requirement and I don’t see a way to get a different one on Lambda. We may be able to add support for somethign like https://github.com/coleifer/pysqlite3 in the future
a

Anna Geller

07/06/2022, 10:10 PM
@Tomás Emilio Silva Ebensperger how did you install Prefect in your Lambda environment? there are many ways to ship dependencies for Lambda, I'm sure we can find a way that works well
t

Tomás Emilio Silva Ebensperger

07/06/2022, 11:54 PM
running python 3.9
python3.9 -m venv env
then
pip install -U "prefect>=2.0b"
i copied that from the website, the -U is redundent here due to the virtual envornment
a

Anna Geller

07/07/2022, 12:43 PM
I see, and how do you package/deploy it? do you run it in a way that points to Cloud 2.0? otherwise using Prefect doesn't seem too helpful right? because otherwise, you are effectively running an orchestration layer on demand which dies together with each Lambda execution, preventing you to track those runs and use that for observability/tracking history etc
not sure what are your timelines but creating an AWS Lambda recipe for Prefect 2.0 is on my Radar, not sure if I can get that done in July, maybe August but if you're willing to wait a bit, there will be a Discourse recipe here in the near future https://discourse.prefect.io/t/event-driven-workflows-in-prefect-2-0-serverless-workflo[…]ed-based-on-some-action-rather-than-based-on-schedule/1054
t

Tomás Emilio Silva Ebensperger

07/07/2022, 1:20 PM
Thank you Anna. Super helpful. I was running a very similar architecture with prefect 1.0 successfully but the issue was that it was impossible for me to send to the cloud the execution of that particular run, it seemed that you always needed an agent running to see things reflected in the Cloud UI. With prefect 2.0 it is much nicer because i see the local execution runs in the cloud. (for deployment i have used zappa and the serverless framework) it is a bit painful to do everything manually (roles, policies etc). I have a few ideas in the meantime and i will keep learning prefect 2.0, there might be something i am missing. I am fairly new to Orion.
🙌 1