https://prefect.io logo
h

Harry Baker

07/19/2021, 8:47 PM
is there way to pass in a parameter for retry_delay in a task? i have a task that imports google sheets, and want to set up some kind of exponential backoff to my attempts whenever i run into the API limit.
k

Kevin Kho

07/19/2021, 8:49 PM
Hey @Harry Baker, unfortunately we don’t have exponential backoff. Are you on Cloud? You could use the task concurrency limiting and set it close to the API limit.
h

Harry Baker

07/19/2021, 8:52 PM
i'm probably gonna end up running everything from a local server, but might look into that. would it cause issues to have try/except logic within a task to create escalating wait times?
k

Kevin Kho

07/19/2021, 8:59 PM
It wouldn’t…but how would you go about keeping the state for the next exponential retry?
h

Harry Baker

07/19/2021, 9:02 PM
yeah this is probably just gonna cause more issues. the amount of data i'm working with isn't that big so i can just throw in a couple time.sleep() calls to avoid hitting the API limit until i can figure out a more elegant solution
k

Kevin Kho

07/19/2021, 9:03 PM
Yeah a random integer with
time.sleep(2**x)
would work.