hello, some questions about orion (I am new so I decided to start with orion):
• is there a way to run it on a fargate cluster?
• is there a complete example of using own ssh cluster?
• when I run things from jupyter, is this scenario considered supported?
• if I change something in the package I develop - how do I make the same change delivered to agents/executors?
n
nicholas
10/11/2021, 7:49 PM
Hi @Alex Rogozhnikov - since Orion is still in beta some of the more intricate deployment stories aren’t fully mapped; if you’re looking for a production-ready system we recommend using Core 0.15.x and Prefect Server/Cloud, otherwise we recommend using Orion as a local development tool
a
Alex Rogozhnikov
10/11/2021, 8:24 PM
hey @nicholas, thanks for advice.
👍 1
Alex Rogozhnikov
10/11/2021, 8:30 PM
@nicholas
So in the land of prefect-core:
I see fargate depracated in the favor of
ECSAgent
, and probably I can set up dask cluster over SSH.
Didn't find any specific mention of running/developing from jupyter. Can you comment on this?
n
nicholas
10/11/2021, 8:38 PM
Are you thinking to run jupyter notebooks within your flow or are you wanting to write your flows in a notebook and execute them from there?
a
Alex Rogozhnikov
10/11/2021, 8:39 PM
latter
n
nicholas
10/11/2021, 8:49 PM
Got it - for tutorials on that you should check out
this livestream▾
with Prefect’s CTO Chris White; you should be able to run flows as expected with Jupyter 🙂
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.