https://prefect.io logo
Title
r

Russell Brooks

10/05/2021, 5:59 PM
Ok, so Orion is next up...Some of Prefect’s negative/positive engineering ands the concept of supervisor reminds me of what I liked about the erlang programming language and it’s philosophy to just fail and to supervise. At any rate, I’ve been using it for around 2 months now and find it relatively straight forward. I’d like to see more worked examples and variety of examples. The docs are well written but need more examples. Also I’m having a nightmare to get it off my localhost and “deployed” not clear how and where I need to set everything up, e.g. runconfig vs storage vs server ui and so on. How many moving parts to get a hello world deployed remotely but within my institution’s architecture?
k

Kevin Kho

10/05/2021, 7:34 PM
Hi @Russell Brooks, about the examples, we sure could use some more for sure. What are you specifically looking for? There are quite a number of combinations though you can mix and match with Prefect. There are also reasonable defaults that you can introduce the more complex components one by one. Are you using Server or Cloud? Cloud has 10000 free task runs and is a lot easier to get started with
r

Russell Brooks

10/06/2021, 4:22 AM
Server. With regards to examples, whichever specific would help me will change each day during the learning phase. So rather than I give you too many specifics, just work to get more examples across the board. If I search the web I don’t find too many.
Right now I have struggle for weeks to try and figure out which bits need to be set and how to set them to get my current Prefect flows and tasks out of localhost and onto my institution’s proper production environment. So more examples of people doing that would be today’s specific.
k

Kevin Kho

10/06/2021, 4:26 AM
There is this blog post how to set server up on a compute instance. Just note that there was a fairly recent change that requires it to be started with
prefect server start --expose
instead of just
prefect server start
since 0.15.5 to allow for outside connections.
r

Russell Brooks

10/06/2021, 4:26 AM
Also lots of your pre-rolled tasks, e.g. mysql. I don’t use as they are too atomic. And too many of my tasks are one liner functions — so lots of code bloat to force it into a format to make it work in Prefect. I hope Orion helps with its subtasks in that regard. Or worked examples on how other people have handled more intermediate or complex use cases.
k

Kevin Kho

10/06/2021, 4:27 AM
You certainly don’t have to use the task library if it doesn’t fit your use case. A lot of people also subclass the tasks there and tweak the
run()
method to adjust it for their needs.
What is your production environment? Would the flows run on ECS? Docker? Kubernetes?
You can see Anna’s latest post here on ECS