OK here we go I noticed that the UI of Orion is way back beh

Question

OK here we go I noticed that the UI of Orion is way back behind the funtionality of Prefect When can one expect a catchup

Kevin Kho · Answer

Hey < volkerjaenisch> keeping the discussion to Slack threads would be preferred since it helps us revisit the message as more messages come in We ll still see the updates

volkerjaenisch · Answer

Oh sorry Using slack to first time Was the problem that I addressed you directly

Kevin Kho · Answer

Not at all I am responsible for the community so feel free to ping me Just pinging other Prefect employees is discouraged

volkerjaenisch · Answer

Please help me out Where exactly shall I post my questions to Obviously not this thread

Kevin Kho · Answer

This thread definitely works You can ask everything here

Kevin Kho · Answer

If you want to discuss them as separate issues could we at least move code blocks and tracebacks to the respective issues so they don t crowd the main channel

volkerjaenisch · Answer

So sorry Can you please guide me more

volkerjaenisch · Answer

I think I am not allowed to move a message between threads

Kevin Kho · Answer

You can just edit a post copy everything and paste it into a thread

volkerjaenisch · Answer

OK I will try

volkerjaenisch · Answer

Is there a plan to include class decorators flow or instance method decorators task Background is that we have lots of scientific code already organized in a strict Factory Worker pattern It seems possible to shape our model with a simple adapter decorators to work with orion And that is quite cool Will this stil be possible in the future of orion Please keep an eye on this usecase `from prefect import task` `def to func cls ` ` Decorator to transform an operator class to a` `function for an orion task decorator ` `instance = cls ` `return instance run` `class BaseWorker ` `def out self todo ` `print todo ` `def run self todo ` `self out todo ` `return todo` ` task name= WorkerA ` ` to func` `class WorkerA BaseWorker ` `def out self todo ` `print Worker A + todo ` ` task name= WorkerB ` ` to func` `class WorkerB BaseWorker ` `def out self todo ` `print Worker B + todo ` `class WorkerFactory ` `def call self choice ` `if choice == A ` `return WorkerA` `elif choice == B ` `return WorkerB` ` flow ` `def main ` `worker = WorkerFactory A ` ` worker Examine Orion ` `worker = WorkerFactory B ` ` worker Examine Orion ` `if name == main ` `main ` Cheers Volker

volkerjaenisch · Answer

Better

volkerjaenisch · Answer

To get more feeling of the development process of Orion I tried to install the dev Version from GH This ends up with ` orion 2cnkrIXd py3 9 volker runner ~ workspace ORION orion$ prefect orion start` `Starting Orion API ` `INFO Started server process 471043 ` `18 03 22 344 | Started server process 471043 ` `INFO Waiting for application startup ` `18 03 22 344 | Waiting for application startup ` `18 03 22 344 | Agent service scheduled to start in app` `18 03 22 344 | Scheduler service scheduled to start in app` `18 03 22 344 | MarkLateRuns service scheduled to start in app` `INFO Application startup complete ` `18 03 22 348 | Application startup complete ` `INFO Uvicorn running on Press CTRL+C to quit ` `18 03 22 349 | Uvicorn running on Press CTRL+C to quit ` `18 03 22 386 | Scheduled 0 runs ` `18 03 22 387 | Finished monitoring for late runs ` `18 03 27 370 | Finished monitoring for late runs ` `INFO 127 0 0 1 45934 OPTIONS graphql HTTP 1 1 200 OK` `INFO 127 0 0 1 45934 POST graphql HTTP 1 1 404 Not Found` `INFO 127 0 0 1 45934 POST graphql HTTP 1 1 404 Not Found` `INFO 127 0 0 1 45934 POST graphql HTTP 1 1 404 Not Found` `INFO 127 0 0 1 45934 POST graphql HTTP 1 1 404 Not Found` `18 03 32 370 | Finished monitoring for late runs ` `INFO 127 0 0 1 45934 POST graphql HTTP 1 1 404 Not Found` `INFO 127 0 0 1 45934 POST graphql HTTP 1 1 404 Not Found`

Kevin Kho · Answer

Thank you

volkerjaenisch · Answer

Orion lags behind at some crucial things like logging Even the local logging does not work with the a4 version When are such basic operations planed to be available Will the UI of Orion match the UI of Prefect and if when

volkerjaenisch · Answer

Sorry my fault Got a bit lost in the UI

Kevin Kho · Answer

No worries at all

Kevin Kho · Answer

On logging and the UI they will take different forms from current Prefect We don t have any public roadmap or timeline for them If you need something production ready the recommendation would still be to the current Prefect not Orion

Kevin Kho · Answer

I will pull someone for the other questions Responses may be slow as US holidays are nearing

Kevin Kho · Answer

I think the people who can answer are out of the office today already so do expect a response tomorrow and feel free to ping me if you don t get one

Kevin Kho · Answer

On the `dev Version` from GH it looks like you are running Prefect Server on the same machine and it is occupying the port

volkerjaenisch · Answer

We will need the functionality flexibilty of Orion Prefect is out of question since Prefect does not allow for the generation of Flows at runtime Prefect is absolutely cool for the KI people out there running really simply workflows We are dealing with a scientific code with 280 usecases Not 280 codes but one code with so many usecases It is one of the scientific kernels for the ACTRIS SCC code which evaluates and derives the LIDAR data for Europe Disclaimer This is not the customer I talked about formerly Their code is much more complex We have a deep nested graph of arrows and boxes Every box is an operation class and every arrow is a factory to produce the worker class algorithm chosen by the user So the code is statically a DAG and for sure a DAG as any code at runtime But the second requirement to this code is that any part Box of it may be replaced at runtime by a replacement Plugin This requirement has lead our code design to a component approach ZCA We are absolutely happy with our design Orion will not be the hero to rescue the damsel in distress But we had planned to come up with a monitoring system like Prefect for this code Prefect will not do the job since we have nested Flows at runtine Orion is exactly what we need

volkerjaenisch · Answer

Thank you for the hint with the occupied port This may be quiet possible And this should be shouted by Orion into my face

Zanie · Answer

Sounds really cool Logging in Orion is actually far more powerful you can configure individual logging classes custom formats etc we just haven t decided how we want to store the logs server side We re in the process of designing that right now I m not sure about a class decorator Classes are stateful and I wouldn t want to mislead users as state will behave differently as concurrency and distributed execution is introduced You can use a class to return static methods wrapped as tasks If you re just using them as composable function factories and not making use of state you shouldn t need a class level decorator

volkerjaenisch · Answer

Yep Thank you so much The occupied port was the issue

Zanie · Answer

With the release of the schematics today in the UI we re moving quickly towards parity there Much of what s missing from the UI doesn t exist server side yet As we introduce things server side e g run configs we ll be adding them to the UI as well

volkerjaenisch · Answer

< Zanie> I agree with you all the way The first thing we smash into our customers brains is to never store state in an operational class We have a strict seperation of data classes and operational classes Op Classes modify data classes Data classes are dumb but may have properties to convert unit and so Without this you will never be able to do easy unit testing pickle before pickle after of a certain part of code I am not strictly asking for a class decorator You are absolutely right this may mislead I just beg you to keep in mind people like us having a well structured component designed code heavily armed with OOP that also like to participate on your truly glorious way to Orion I volunteer to be a tester on this part of Orion since it is so COOL Thank you so much for Prefect and Orion

volkerjaenisch · Answer

Another few questions I would like to ask 1 Since in Orion there is no graph know before the process is running a How do you visualize the graph structure b How do you optimize e g parallelism 2 When is a feature freeze to be expected Aka a beta Version 3 Will the overall Operation of Orion remain so lean I appreciate not having Docker containers 4 Will you offer support for self hosted Orion installations 5 Are there known limitations to Orion

Zanie · Answer

1a There is an intention to investigate a dry run mode for visualization of the graph before runtime Otherwise see 1b Parallelism is eager right now Since the DAG is dynamic you can actually control the parallelism more concretely We ll be certain to ensure people can do things like depth first execution 2 3 Yes we do not want to ask users to use docker compose to run the server 4 I cannot promise but we have offered support of this manner in the past 5 I m not sure yet We re still learning You d have to be more specific I suggest reading the rest of the FAQ as it covers some of these things

volkerjaenisch · Answer

Thank you for the fast and thoroughly answers Today the decision was made to give Orion a chance Followed up by prefect as plan B

volkerjaenisch · Answer

But another question comes around Prefect offered a functional as well as an imperative approach Will there be support for an imperative approach for Orion as well We would like that really much