Hey Y'all! I'm running a self-hosted server and ECS worker in AWS (ECS). I'm running into a rate limit from AWS on the
DescribeTasks
operation.
Copy code
botocore.exceptions.ClientError: An error occurred (ThrottlingException) when calling the DescribeTasks operation (reached max retries: 4): Rate exceeded
I'm guessing this happens on the worker/server as it tries to poll for the status of the ECS task running the Flow.
Does anyone have any guidance on how to deal with this? Is there an env var that I can use to turn down how often it polls for status?
Jonathan Samples
02/20/2025, 8:49 PM
🌩️ Just read through the prefect code a bit and found the job configuration property:
task_watch_poll_interval
It defaults to 5s. So I'll just set it higher.
b
Bianca Hoch
02/21/2025, 9:27 PM
Hey Jonathan! Just checking in. Did you have any luck adusting the poll interval?
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.