Hello. This is related to the questions asked by ...
# prefect-server
Hello. This is related to the questions asked by @Christopher Chong Tau Teng I am trying to understand if Prefect Server can be used in our environment to manage multiple workflow streams. We want to manage workflows in the following manner 1. Workflows are containerised 2. You can update the container image backing one workflow without affecting any other workflows 3. Some workflows depend on other workflows having completed correctly 4. Prefect itself is deployed in containers, not on bare-metal host Call a container image which is to be used by one or more workflows a workflow engine. My current understanding on how to do this is to separate out the flow scripts from the workflow engines, but I am a bit confused as to how I should actually implement this. As a template example, consider a setup with * Prefect Server running as as set of docker containers * Two engine container images, tagged engine:1 and engine:2 respectively * Two workflows WF1 and WF2 with asscociated scripts workflow_1.py and workflow_2.py Then, the scenario plays out as follows: 1. both workflow_1.py and workflow_2.py are initially using engine:1. 2. an "issue" is identified with WF1 using engine:1. It has to be updated to use engine:2 3. workflow_2.py is updated to use engine:2 and re-registered with Prefect 4. WF1 now runs using engine:1 while WF2 runs using engine:2 How might this setup be configured in Prefect?
@Cormac Long it looks like KubernetesAgent is what you need. Have a look at the docs here and regarding Server deployment, we have a Helm chart with README description showing how to use it. I still recommend giving Prefect Cloud a try first (we have an incredibly generous unlimited in time free tier), at least to test everything out. It’s much easier to get started and you can use Cloud’s out-of-the-box functionality to test out all your workflow patterns, agents, etc. before committing to anything. Then you can decide whether you need to deploy Server or stay with Cloud for production code,
@Anna Geller thanks. we'll have a look at this and see if we can make it work.
🙌 1