let me know if this is not the best place to ask t...
# best-practices
let me know if this is not the best place to ask this. But I’m curious how I might go about dynamically adjusting parameters on retries? The use case is: • processes taking a sum of a large number…if it overflows…I can try taking the sum with a divisor factor • for my use case I just need a number to compare to another - which would have same divisor factor applied. • if it fails again - I can try a larger divisor factor - up to 3 times - and then fail after the nth retry
I wonder whether this is a data flow or rather the logic of your algorithm. How would you write it in Python without Prefect?
I would prob 1. capture the specific error 2. Augment a counter 3. if counter is <=4 4. increment the divisor factor by 10 and resubmit the query
Doesn't really answer the question but couldn't just start with a relatively high divisor factor in order to avoid errors ?
upvote 1
Marco, from Prefect 1.0 perspective, you may try implementing that using looping and in Python 2.0 you can write arbitrary Python so no limits here in Prefect 1.0 an interesting approach is that now you can use
on your task decorator to allow tasks to retry only on a specific exception type - might be exactly what you are looking for - #5634
Thanks everyone! I def could try to start with a bigger number - but if the sum turns out to be a pretty small and its non-integer data types then it just doesn’t work so well. But I’ll give that a look over. Thanks!
👍 1