Morning, quick question does prefect have to be in...
# prefect-community
Morning, quick question does prefect have to be installed where the agent is running from? If so does it not create 2 database instances?. When prefect is installed on host running the webservice the .prefect directory and orion.db instance gets created. And then when installing prefect again on the host from where the flows are being run then another .prefect directory and orion.db instance gets created?
Hello Byron, in short, yes. Prefect must be installed in any environment in which you intend to run the agent or execute a flow run.
In regard to whether or not it creates a new instance, let me consult with the team and we'll get back to you here.
Hi Bianca thanks for the reply. I hope you dont mind, but I think this thread might get a bit lengthy as I have lots of confusion around the setup and documentation. I don't know if there are concepts introduced in Prefect V1, but I'm starting from the documentation on Prefect V2. The context I have are just based on what I'm reading within this documentation. Example for work-queue setup, it speaks to agent configuration, but its not clear how the agent is configured. The backend seems to expect certain fields that describes the concept of an agent, but I cannot seem to find what that is from the PrefectV2 documentation. From my screenshot it looks like "prefect work-queue" CLI command has to run first in order to create the entry on table "work_queue" which is then a key into the "agent" table (work_queue_id), but I can run "prefect agent start" command even though the agent on table "agent" does not exist yet. I need to understand how to configure the agent first and only then it should allow me to start the agent schedule. The command "prefect agent start <queue-id>" is also confusing. When the agent is created it should already be associated with a queue, and the command prefect agent start <agent-id> make more sense. Please let me know your thoughts
this is something in addition to the multiple environments that require prefect to be installed. There is the agent process that could run on hostA. There is the execution environment where the agent deploys the flow run which also requires prefect to be installed (lets say on hostB), then there is the orion service (webservice) that runs on a hostC that also requires prefect to be installed. In each of these cases the .prefect project directory gets created, and I'm confused as to why and what prefect environment variables to set for each of these hosts. Will continue to discover in the meantime while I await feedback.
Hi Bianca, some feedback on the definitions used. The term "concurrency_limit" is used in two different configurations. It refers to both queue concurrency and task concurrency limits which is confusing. My suggestion here is to use task_concurrency_limits for tasks and also rename the database tbl and prefect CLI command accordingly (+ doc update) and the concurrency_limit field on the work_queue table to remain as is, since it is an attribute of the work-queue object and makes sense when configuring it
Hi Bianca, I notice when a flow is configured (within your code base) and then run the flow manually with (python it will automatically populate the table "flow" on the backend. However the DB model object does not align with what can be configured from the documentation explaining the flow concept. image below shows the configuration vs the backend
From the documentation these configurations can be set, but where does it persist?