https://prefect.io logo
t

Thomas Fredriksen

02/04/2022, 12:06 PM
Hello everyone. We have a prefect-server instance running in our cluster, and we were wondering what would be the best way to organize the projects? Is it possible to create directory-like structure of the form
{DOMAIN}/{PROJECT_NAME}
(or longer) or similar? For example
iot_devices/site-temperature-management
a

Anna Geller

02/04/2022, 12:09 PM
When you say projects, do you mean the Prefect 1.0 concept of projects or rather how you can structure e.g. a Github repository for flows?
t

Thomas Fredriksen

02/04/2022, 12:10 PM
prefect-projects, yes
a

Anna Geller

02/04/2022, 12:10 PM
Btw, Orion will make it much easier - check out this page for more details https://discourse.prefect.io/t/how-can-i-organize-my-flows-based-on-my-business-logic-or-team-structure/66
t

Thomas Fredriksen

02/04/2022, 12:11 PM
Cool! But is this possible in Prefect 1.0?
a

Anna Geller

02/04/2022, 12:14 PM
No, unfortunately not. Currently, projects don’t allow a nested structure as you described. So you could not have one project called
iot_devices
and a child project of it called
site-temperature-management
. But in Orion you could have nested tags:
Copy code
>>> @flow
>>> def my_flow():
>>>     with tags("a", "b"):
>>>         with tags("c", "d"):
>>>             my_task()  # has tags: a, b, c, d
>>>         my_task()  # has tags: a, b
t

Thomas Fredriksen

02/04/2022, 12:16 PM
Awesome to hear Orion reaching maturity. Really looking forward to see it graduate from beta
👍 1
a

Anna Geller

02/04/2022, 12:16 PM
So in 1.0 I would probably try to bring more structure in the code organization. E.g. you can still have a project called iot_devices, and you can have e.g. a Github repository for this project and this repository may have directories corresponding to each project
not sure what Git platform are you using, but many allow organization through projects or namespaces (e.g. Gitlab, Bitbucket)
t

Thomas Fredriksen

02/04/2022, 12:17 PM
we are already doing that, but the Web UI is getting a bit crowded
a

Anna Geller

02/04/2022, 12:19 PM
Gotcha. Doing something like this would be hard in 1.0 but Orion will definitely make this easier
t

Thomas Fredriksen

02/04/2022, 12:20 PM
Perfect, thank you!
👍 1
2 Views