https://prefect.io logo
m

Matthias Roels

02/17/2022, 1:07 PM
I have a question regarding Orion that is confusing me. With Prefect 1.0, we have the concept of flow storage. Is there an equivalent for Orion or how does it work? Say I want to run a flow on k8s, can I just package my flow code in a Docker container along with the required dependencies (maybe using the prefect image as base image)? Thanks for clarifying!
For example in this tutorial https://orion-docs.prefect.io/tutorials/kubernetes-flow-runner/, is the flow code just stored on your local machine and mounted in the container at run-time?
a

Anna Geller

02/17/2022, 1:35 PM
Good question. We are currently working on the storage abstraction. As of now, you can use local storage when you specify
flow_location
on your DeploymentSpec. When you use
DockerFlowRunner
, Prefect will create a Docker container for the flow run and will pull the flow from Orion backend. Similarly when you use
KubernetesFlowRunner
, Prefect will create a Kubernetes job for the flow run and at runtime will pull the flow from the backend. When you create a deployment, you can specify the flag
push_to_server
which is by default set to True. This means that by default, when you create a deployment, the flow text will be loaded from the flow location and stored on the Orion server instead of locally, which allows the flow to be compatible with all flow runners. So in short, by default when you create a deployment, Prefect will store the code in your Orion server, making the process much easier so that you don't need to worry about the storage as much as you had to in Prefect 1.0. Having said that, we are actively working on all that and the best way to stay up-to-date is to watch the #announcements channel and in the coming weeks you will see more updates on that for sure.
m

Matthias Roels

02/17/2022, 2:06 PM
That makes a lot of sense! I will continue watch the channel to look out for updates. Look forward to a production-ready release of Orion so that I can start experimenting with it to replace Argo Workflows 😀
🙌 1
Maybe one last question, with the flag
push_to_server
set to True, does that mean that your flow code gets uploaded to Prefect Cloud when using that offering?
a

Anna Geller

02/17/2022, 2:16 PM
I don't know how this will be handled in Cloud 2.0. I'm pretty sure you will have a lot of flexibility allowing you to choose the right storage method based on your preferences and security/privacy requirements
👍 1
11 Views