William Jamir08/04/2022, 4:37 PM
Khuyen Tran08/04/2022, 6:02 PM
William Jamir08/04/2022, 9:44 PM
You can check which work queue the deployment is executedYou mean is going to be executed, right? Because this tab shows only the upcoming Runs. In my case, both work queues (Test1 and Test2) have the same runs. But, from the output of the CLI, it looks like all the tasks goes to only one “Work queue”, is that the expected behavior? Besides that, is there a way to check which “Work Queue” the task was executed?
B) Start an agent for each “Work Queues” on theses servers?
agent start 42 # On machine staging-1 agent start 42 # On machine staging-2
Now, if I want to limit to 10 parallel runs on each server, I need to create a “Work Queue” for each server, using the same tag, right? But from my first image, looks like the runs it’s not correctly “dispatched”, am I doing something wrong?
agent start 41 # On machine staging-1 agent start 42 # On machine staging-2
Khuyen Tran08/05/2022, 3:20 PM
William Jamir08/05/2022, 3:45 PM
Yes you can only see the upcoming runsAh okay, thank you for the head-up 🙂
Do you mean two work queues have the same tag?Yes, in my understanding from reading the docs, this was the correct way to distribute runs on different servers 😬
Is there a reason why you want to check which work queue a task was executed on?I had issues in the past (with Prefect 1.0) where I had jobs failing only in one agent (I have four agents). So, this feature from Prefect 1.0 helped me to identify easily an issue with my infrastructure, and I was hoping to have something similar on Prefect 2.0 in case another issue like that popups again 🤓
If you have two agents of the same work queue, Prefect will pick only one agent and run the flowAh perfect! Thanks for the clarification.
I don’t think this will help distribute runs on different servers. But I’ll check with the teamThanks! I really appreciate any input on how to approach this configuration 😊 Just as disclosure, my agents are executed on-premise servers (without docker), this is why I have this concern about the history of executions and the identification on which server the run is being performed ;)
Khuyen Tran08/05/2022, 3:48 PM
Yes, in my understanding from reading the docs, this was the correct way to distribute runs on different serversCan you point me where it is in the docs?
Just as disclosure, my agents are executed on-premise servers (without docker), this is why I have this concern about the history of executions and the identification on which server the run is being performed ;)Ah got it. Thank you. I’ll talk to my team about this
William Jamir08/05/2022, 4:13 PM
Can you point me where it is in the docs?Sure, it was not a single point on the docs, but rather a general interpretation from the section. The first attachment, with the red circle, made me understand that I need multiple Agents for a single “Work Queue.” But if I do that, only one agent is consuming all the runs (as shown from the second screenshot) So, since this approach was not working, I concluded that this section generally referred to the queues. Then I tried to have multiple queues with the same tag (no place over the docs mention that this is prohibited or wrong) and start an agent for each queue. Which doesn’t work either
Khuyen Tran08/05/2022, 6:35 PM
will run deployment with the tag
. Running multiple work queue with the same tag will not allows you to run your flow across work queues, but we will consider about it in the future.