Hi general question here When running flows lately I have been noticing lately long waits for workers, hour+, and when I do happen to get a worker upon creation of a cluster its only ever one worker. Is this common? I tried just creating coiled clusters and they have no problem getting workers. Any help is appreciated thanks.

Hey @Michael Welsh, we have seen a few mentions of this but we checked our latency of requests over the past week and didn’t find anything out of the ordinary. Do you know if this happens with anything not involving a cluster spin up like a LocalRun or DockerRun? 🤔

I can create clusters using coiled api and get workers.

Are you still seeing the long waits? Do you have spot instances turned on with Coiled? Also when you posted this, I think they had high memory utilization that day and did a restart.

no I don't have spot instances on. I have 4 runs today and haven't gotten a worker yet.

Do your logs show anything and do you see it trying ti spin up a cluster on the Coiled dashboard?

yeah so it spins up a cluster on the dashboard, it only show s 0/4 workers only and my flow runs waitn g for a worker

I think this might be more of a Coiled thing than a Prefect thing. They may be able to tell you why you weren’t able to get workers

ok I can' checkin with them

Hey @Michael Welsh, I think you might be fine but I just wanted to tag @Greg Smith here so he can keep track

Yes, please LMK if this persists. I'd like to see if I can reproduce it and get it addressed.