Hi, I'd like to validate my assumptions about queue priorities and if I'm dealing with a bug.
Following scenario:
1. Prefect v2.17.1, a single worker for all queues on a single worker pool
2. three flow runs are scheduled, all on a queue with priority 2
3. the first of these runs is currently running on the worker, the other two are waiting in the queue
4. a fourth flow run get scheduled, but on a queue with priority 1
5. the first run finishes
I would expect the fourth run to be picked up by the worker now, because it is on a queue with higher priority (lower number). The other two schedule runs from the first batch would only run after once the higher priority queue is empty again.
Is that behaviour expected and correct?
If I don't observe that behaviour, would could be the issue?
The described behaviour is reproducible in my local setup, but I have observed in production that pending flow runs on a lower priority queue (higher number) would run prior to high priority flow runs that got scheduled.