hello Prefect team, hello community, is there more information available on how to make the ray autoscaler and prefect 2 task concurrency limits work smoothly together? currently, when i submit a large number of tasks with a concurrency limit tag, the ray autoscaler will see the large amount of tasks and scale accordingly, however only a small number of tasks is allowed to run concurrently due to tag limits and most of the upscaled resources won't be used, which causes unnecessary costs especially for tasks with high resource requirements. i think this is due to the fact that the concurrency limit check happens during propose_state which happens inside the call that is submitted to the task runner. is there any way how this can be avoided? thanks a lot for any input.

Hi Justin. If I understand your question correctly, you are trying to avoid Ray scaling up lots of unnecessary resources. You would like the concurrency limit check to occur earlier, is that correct? If so, I do not know of a way to achieve that. I suggest you open a feature enhancement issue on GitHub.