Hi Everyone, I am running 3 different deployment w...
# ask-community
n
Hi Everyone, I am running 3 different deployment with 3 different queues. if there any constraints when all 3 queue were at peak load. we are experiencing a issue where process in "q1" in getting slower when "q2" and "q3" is running. How can i optimize this. All three queues has concurrency 10
d
what is
a issue where process in "q1"
?
n
There is 1 task which is we are sending file using the Dicom node ( in genral POST request ) and this sending file takes around 1:30 min Whereas, when the q2 and q3 is ideal then the sending time reduce to like 30seconds
d
I extensively use queues and have not faced such issue. Are you using Cloud or Oss?
n
We are using cloud AWS
d
I mean the prefect server πŸ˜„
self hosted prefect or cloud ?
n
Senf hosted
Self*
d
I dont think your flow code has any interactions with the queue status. Id try the following: 1. Check over all mem usage on the machine [where server(s) and / or workers / agents are running] 2. Check the health of database 3. Run worker connected to q1 on a seperate machine and see if the issue persist
n
I am using aws ec2 with auto scaling, do you think memory will be a issue ?
d
Depends on how the scalig has been configured i guess, I dont use aws πŸ˜„
How are u measuring the time difference on file upload? Have you tried adding logs right before and after the request.post to time the request completion? Or are you assuming the request post time based on a prefect task completion time?
n
Both, the prefect task completion time and manually too (for manual i have use python β€œtime” module
d
Thats indeed weird. I would assume there is no relation between your request code and the queue status. I guess its time for some prefect dev to weigh on it
n
Yeah, hopefully 🀞