Hi everyone,
What’s the recommended approach in beta to splitting up alpha/beta/gamma environment? We’re in beta so there is a single workspace per email address that’s shared (perfect@someco.com), so we can’t do different envs per workspace. One thought was a single workspace, with a single S3 storage (storage cli is so weird BTW, no ability to remove a storage) with multiple work queues. Then there would be one agent per env, each consuming a different work-queue for some isolation? Then use tags to separate which flow runs go to which work queue? What are the recommendations in terms of architecture here?
k
Kevin Kho
05/22/2022, 5:44 PM
You will be able to do multiple workspaces, each representing a different env eventually. The storage interface will change. Even with multiple workspaces, the work queues and agents are currently scoped to a workspace so you would just make a work queue for each workspace.
If you need something now, I think it would be what you suggest with multiple accounts, each with their workspace, work-queue, and agent
a
Anna Geller
05/22/2022, 6:22 PM
@Jan Domanski I'd suggest you can start now with a local Orion environment as a sandbox/dev environment and Cloud 2.0 as your prod. You can use
prefect profile use cloud/dev
to switch between those.
Workspaces feature is a high priority item, so watch the #CKNSX5WG3 and release notes on Discourse
j
Jan Domanski
05/22/2022, 8:18 PM
That's awesome thank you! But is there a way to get it to work with 3 separate work queues and tags? Obviously the isolation would not be ideal
Jan Domanski
05/22/2022, 8:21 PM
The dream scenario for me is using tools like Pulumi to declare the desired infra state on prefect (workspaces, queues, flows)
a
Anna Geller
05/22/2022, 9:15 PM
This should be doable - if you do it on some VM, it would probably be a matter of some bash script magic. Not to say it's easy but definitely something you may already give a try 👍
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.