Hey Prefect I am trying out the new `push workpool` for ECS

Question

Hey Prefect I am trying out the new `push workpool` for ECS I noticed when I deploy and run a flow I can still see a `work queue` assigned to the flow deployment although in the new work pool there doesn t seem to be a `work queue` tab anymore Are there still queues or is this something being removed in a future release

Ben Muller · Answer

update when I click on the queue link I am able to navigate to the queue page which seems to be semi permanently in a `unhealthy` state

Ben Muller · Answer

but trying to navigate there from the actual pool page is not possible

Ben Muller · Answer

I am guessing it is unhealthy because the queue is expecting a worker to poll it or something so without this there is some default behaviour assuming it is unhealthy

Jenny · Answer

Yes Changes for the UI are in the works <https github com PrefectHQ prefect ui library pull 1603> And you are correct that it s unhealthy because push work pools don t have workers to poll them We ve hope to add some updates to add status or health for push work pools but it s a little more complex

Ben Muller · Answer

Thanks < Jenny> I am testing out how much horizontal scaling these pools can handle when I run about 20 or more flows at a time the flow runs seem to bank up and enter `Late` states for a few minutes All my concurrency is set to unlimited I was hoping this would be really easy to scale huge amounts is that kind of thing the expectation

Ben Muller · Answer

my concern is that moving away from workers work queues I was able to prioritise flow runs by having workers poll queues exclusively Thus ensuring that the high priority flows always run Without workers and assuming there is limited concurrency scaling with the push work pools there is no way to prioritise flows without creating new work pools in the same way you would use queues but this doesnt seem like the expected design

Jenny · Answer

I see Jake typing and he s the real expert so I ll let him reply

Jake Kaplan · Answer

hey < Ben Muller> thanks for trying out push pools At the moment you are right they don t support priority via queues We re evaluating if it s something we want to introduce Ideally we re looking offer push work pools at scale so that prioritization is not a primary factor for users we re not ruling it out though They re still very much being worked on in beta A few minutes is definitely not the expected latency time for your flow run to be picked up Would you mind sharing your workspace id feel free to dm if you d rather and I d be happy to take a look