<@ULVA73B9P> how can add some jitter to the start ...
# ask-marvin
m
@Marvin how can add some jitter to the start of a bunch of parallel tasks kicked off with map
m
To add jitter to the start of parallel tasks kicked off with
map
in Prefect 3.x, you can utilize the
retry_jitter_factor
option. This option introduces a random amount of time to the retry periods, helping to prevent "thundering herd" scenarios where many tasks might start at the same time. Here's a conceptual approach: 1. Use the
retry_jitter_factor
to add variance to the base delay time. 2. Set a retry delay and apply the jitter factor to introduce random delays. For example, if you have a retry delay of 10 seconds and set a
retry_jitter_factor
of 0.5, the delay can vary up to 15 seconds. For more detailed implementation, you can refer to the Prefect documentation: Prefect Tasks with Jitter.