https://prefect.io logo
n

Nimesh Kumar

08/22/2023, 5:58 AM
Hi Everyone, I am running 3 different deployment with 3 different queues. if there any constraints when all 3 queue were at peak load. we are experiencing a issue where process in "q1" in getting slower when "q2" and "q3" is running. How can i optimize this. All three queues has concurrency 10
d

Deceivious

08/22/2023, 2:10 PM
what is
a issue where process in "q1"
?
n

Nimesh Kumar

08/22/2023, 2:13 PM
There is 1 task which is we are sending file using the Dicom node ( in genral POST request ) and this sending file takes around 1:30 min Whereas, when the q2 and q3 is ideal then the sending time reduce to like 30seconds
d

Deceivious

08/22/2023, 2:16 PM
I extensively use queues and have not faced such issue. Are you using Cloud or Oss?
n

Nimesh Kumar

08/22/2023, 2:18 PM
We are using cloud AWS
d

Deceivious

08/22/2023, 2:18 PM
I mean the prefect server πŸ˜„
self hosted prefect or cloud ?
n

Nimesh Kumar

08/22/2023, 2:19 PM
Senf hosted
Self*
d

Deceivious

08/22/2023, 2:21 PM
I dont think your flow code has any interactions with the queue status. Id try the following: 1. Check over all mem usage on the machine [where server(s) and / or workers / agents are running] 2. Check the health of database 3. Run worker connected to q1 on a seperate machine and see if the issue persist
n

Nimesh Kumar

08/22/2023, 2:24 PM
I am using aws ec2 with auto scaling, do you think memory will be a issue ?
d

Deceivious

08/22/2023, 2:25 PM
Depends on how the scalig has been configured i guess, I dont use aws πŸ˜„
How are u measuring the time difference on file upload? Have you tried adding logs right before and after the request.post to time the request completion? Or are you assuming the request post time based on a prefect task completion time?
n

Nimesh Kumar

08/22/2023, 2:28 PM
Both, the prefect task completion time and manually too (for manual i have use python β€œtime” module
d

Deceivious

08/22/2023, 2:30 PM
Thats indeed weird. I would assume there is no relation between your request code and the queue status. I guess its time for some prefect dev to weigh on it
n

Nimesh Kumar

08/22/2023, 2:34 PM
Yeah, hopefully 🀞