Hello. I’m using ECS Agent and dask executor using fargate cluster to parralize multiple ML model training. When I use dask executor, flow as ECS Task use DescribeTask API to dask scheduler so many time. CloudTrail show API call from Prefect ECS Task as 500/minute. So API call throttling error occur many times. Is there any way to reduce using DescribeTask API call when I use dask executor?

HI Satsuki, The scheduler is responsible for receiving and scheduling all the dask tasks, both to issue and create the task, and to receive the results back to the flow. How many parallelized tasks are you submitting in a flow?

Thank you for the reply. I have 12 mapped tasks and set 4 workers to dask executor. I found that flow(ECS task) kicked by ECS Agent use DescribeTask API so many times when I use DaskExecutor.